Details
-
Type:
Bug
-
Status: Resolved
-
Priority:
Major
-
Resolution: Fixed
-
Affects Version/s: None
-
Fix Version/s: 3.0.0-alpha2, YARN-5355_branch2
-
Component/s: timelinereader, timelineserver
-
Labels:
-
Hadoop Flags:Reviewed
Description
In HierarchicalTimelineEntity, setparent/addChild allows to set parent/child entities at INFO level. The key is an string and value as an object.
Like below, for YARN_CONTAINER entity parent entity set for application.
"SYSTEM_INFO_PARENT_ENTITY": { "type": "YARN_APPLICATION", "id": "application_1471931266232_0024" }
But to use infofilter on entity type YARN_CONTAINER for an specific applicationId, IIUC there is no way to pass object as value in infofilter.
To make easier retrieval either
- publish parent/child entity id and type as string rather that object like below
"SYSTEM_INFO_PARENT_ENTITY_TYPE": "YARN_APPLICATION" "SYSTEM_INFO_PARENT_ENTITY_ID":"application_1471931266232_0024"
OR
- Add ability to provide object as filter with below format like infofilters=SYSTEM_INFO_PARENT_ENTITY eq ((type eq YARN_APPLICATION) AND (id eq application_1471931266232_0024))
I believe 2nd approach will be well applicable for any entities. But I am not sure does HBase supports such a custom filters while scanning a table.
1st approaches will be much easier to change.
Activity
- All
- Comments
- Work Log
- History
- Activity
- Transitions
I found the way to achieve filtering of an object in infofilter. Infofilters uses an JasonParser to parse the value. So this value can be object in the form of Json.
For example : To list all the container entities belongs to app-attempt, query can be
GET /ws/v2/timeline/apps/application_1471931266232_0024/entities/YARN_CONTAINER?fields=INFO&infofilters=SYSTEM_INFO_PARENT_ENTITY eq {"type":"YARN_APPLICATION_ATTEMPT","id":"appattempt_1471931266232_0024_000002"}
Value should be in the form of Json without any space i.e
{"type":"YARN_APPLICATION_ATTEMPT","id":"appattempt_1471931266232_0024_000002"}
This has to be captured in the AtsV2 documentation along with an example which is missing now for infofilters.
Updated the documentation patch for passing an object in infofilters along with an example.
Summary also updated as per the patch provided.
Updating documentation should be fine.
Objects should be represented as JSON. This was missed in documentation.
Looks fine to me
 +1 overall |
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| 0 | reexec | 0m 19s | Docker mode activated. |
| +1 | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 | mvninstall | 7m 7s | trunk passed |
| +1 | mvnsite | 0m 14s | trunk passed |
| +1 | mvnsite | 0m 11s | the patch passed |
| +1 | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 | asflicense | 0m 16s | The patch does not generate ASF License warnings. |
| 8m 23s |
| Subsystem | Report/Notes |
|---|---|
| Docker | Image:yetus/hadoop:9560f25 |
| JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12825993/YARN-5577.patch |
| JIRA Issue | |
| Optional Tests | asflicense mvnsite |
| uname | Linux e94aae3c5765 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / c258171 |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site |
| Console output | https://builds.apache.org/job/PreCommit-YARN-Build/12925/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
Varun Saxena If patch is fine, do you mind committing it?
Sorry I missed this one. Will commit it tomorrow.
ping Varun Saxena!!
Sorry Rohith Sharma K S, I was on leave so didn't commit it. Will do so shortly.
I will commit it to trunk as its documentation related change. Will be brought in ATSv2 branch when we do trunk rebase.
Committed to trunk.
Thanks Rohith Sharma K S for your contribution
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10459 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10459/)
YARN-5577. [Atsv2] Document object passing in infofilters with an (varunsaxena: rev ea29e3bc27f15516f4346d1312eef703bcd3d032)
- (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/TimelineServiceV2.md
Backported to YARN-5355_branch2 as well
+1 for first approach,
But Sangjin Lee , Vrushali C, Joep Rottinghuis, any idea why we choose to store it as object ?