Details
-
Type:
Sub-task
-
Status: Resolved
-
Priority:
Major
-
Resolution: Fixed
-
Affects Version/s: 3.0.0-alpha1
-
Fix Version/s: 3.0.0-alpha2
-
Component/s: yarn
-
Labels:None
-
Target Version/s:
-
Hadoop Flags:Reviewed
Issue Links
Activity
- All
- Comments
- Work Log
- History
- Activity
- Transitions
 +1 overall |
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| 0 | reexec | 0m 14s | Docker mode activated. |
| +1 | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 | test4tests | 0m 0s | The patch appears to include 5 new or modified test files. |
| 0 | mvndep | 0m 9s | Maven dependency ordering for branch |
| +1 | mvninstall | 7m 1s | trunk passed |
| +1 | compile | 1m 31s | trunk passed |
| +1 | checkstyle | 0m 30s | trunk passed |
| +1 | mvnsite | 0m 45s | trunk passed |
| +1 | mvneclipse | 0m 29s | trunk passed |
| +1 | findbugs | 0m 58s | trunk passed |
| +1 | javadoc | 0m 27s | trunk passed |
| 0 | mvndep | 0m 9s | Maven dependency ordering for patch |
| +1 | mvninstall | 0m 33s | the patch passed |
| +1 | compile | 1m 26s | the patch passed |
| +1 | javac | 1m 26s | the patch passed |
| -0 | checkstyle | 0m 28s | hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server: The patch generated 18 new + 6 unchanged - 0 fixed = 24 total (was 6) |
| +1 | mvnsite | 0m 42s | the patch passed |
| +1 | mvneclipse | 0m 27s | the patch passed |
| +1 | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 | findbugs | 1m 12s | the patch passed |
| +1 | javadoc | 0m 24s | the patch passed |
| +1 | unit | 0m 52s | hadoop-yarn-server-timelineservice in the patch passed. |
| +1 | unit | 5m 3s | hadoop-yarn-server-timelineservice-hbase-tests in the patch passed. |
| +1 | asflicense | 0m 20s | The patch does not generate ASF License warnings. |
| 28m 52s |
| Subsystem | Report/Notes |
|---|---|
| Docker | Image:yetus/hadoop:a9ad5d6 |
| JIRA Issue | |
| JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12840082/yarn5925.001.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle |
| uname | Linux 79aa81bdd99d 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 83cc726 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle | https://builds.apache.org/job/PreCommit-YARN-Build/14028/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server.txt |
| Test Results | https://builds.apache.org/job/PreCommit-YARN-Build/14028/testReport/ |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase-tests U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server |
| Console output | https://builds.apache.org/job/PreCommit-YARN-Build/14028/console |
| Powered by | Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
| +1 overall |
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| 0 | reexec | 0m 14s | Docker mode activated. |
| +1 | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 | test4tests | 0m 0s | The patch appears to include 5 new or modified test files. |
| 0 | mvndep | 0m 14s | Maven dependency ordering for branch |
| +1 | mvninstall | 7m 46s | trunk passed |
| +1 | compile | 1m 53s | trunk passed |
| +1 | checkstyle | 0m 34s | trunk passed |
| +1 | mvnsite | 0m 51s | trunk passed |
| +1 | mvneclipse | 0m 35s | trunk passed |
| +1 | findbugs | 1m 6s | trunk passed |
| +1 | javadoc | 0m 28s | trunk passed |
| 0 | mvndep | 0m 10s | Maven dependency ordering for patch |
| +1 | mvninstall | 0m 40s | the patch passed |
| +1 | compile | 1m 34s | the patch passed |
| +1 | javac | 1m 34s | the patch passed |
| -0 | checkstyle | 0m 34s | hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server: The patch generated 18 new + 6 unchanged - 0 fixed = 24 total (was 6) |
| +1 | mvnsite | 0m 43s | the patch passed |
| +1 | mvneclipse | 0m 27s | the patch passed |
| +1 | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 | findbugs | 1m 15s | the patch passed |
| +1 | javadoc | 0m 24s | the patch passed |
| +1 | unit | 0m 46s | hadoop-yarn-server-timelineservice in the patch passed. |
| +1 | unit | 4m 40s | hadoop-yarn-server-timelineservice-hbase-tests in the patch passed. |
| +1 | asflicense | 0m 17s | The patch does not generate ASF License warnings. |
| 30m 27s |
| Subsystem | Report/Notes |
|---|---|
| Docker | Image:yetus/hadoop:a9ad5d6 |
| JIRA Issue | |
| JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12840092/YARN-5925.01.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle |
| uname | Linux bf2aed5af729 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 83cc726 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle | https://builds.apache.org/job/PreCommit-YARN-Build/14031/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server.txt |
| Test Results | https://builds.apache.org/job/PreCommit-YARN-Build/14031/testReport/ |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase-tests U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server |
| Console output | https://builds.apache.org/job/PreCommit-YARN-Build/14031/console |
| Powered by | Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
New patches to address the reported checkstyle warnings
| +1 overall |
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| 0 | reexec | 0m 11s | Docker mode activated. |
| +1 | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 | test4tests | 0m 0s | The patch appears to include 5 new or modified test files. |
| 0 | mvndep | 0m 11s | Maven dependency ordering for branch |
| +1 | mvninstall | 7m 46s | trunk passed |
| +1 | compile | 1m 32s | trunk passed |
| +1 | checkstyle | 0m 29s | trunk passed |
| +1 | mvnsite | 0m 47s | trunk passed |
| +1 | mvneclipse | 0m 31s | trunk passed |
| +1 | findbugs | 1m 2s | trunk passed |
| +1 | javadoc | 0m 26s | trunk passed |
| 0 | mvndep | 0m 9s | Maven dependency ordering for patch |
| +1 | mvninstall | 0m 40s | the patch passed |
| +1 | compile | 1m 35s | the patch passed |
| +1 | javac | 1m 35s | the patch passed |
| +1 | checkstyle | 0m 29s | the patch passed |
| +1 | mvnsite | 0m 43s | the patch passed |
| +1 | mvneclipse | 0m 27s | the patch passed |
| +1 | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 | findbugs | 1m 27s | the patch passed |
| +1 | javadoc | 0m 26s | the patch passed |
| +1 | unit | 1m 9s | hadoop-yarn-server-timelineservice in the patch passed. |
| +1 | unit | 4m 53s | hadoop-yarn-server-timelineservice-hbase-tests in the patch passed. |
| +1 | asflicense | 0m 21s | The patch does not generate ASF License warnings. |
| 30m 24s |
| Subsystem | Report/Notes |
|---|---|
| Docker | Image:yetus/hadoop:a9ad5d6 |
| JIRA Issue | |
| JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12841155/YARN-5925.02.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle |
| uname | Linux 0d8b0e0590bb 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 69fb70c |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| Test Results | https://builds.apache.org/job/PreCommit-YARN-Build/14135/testReport/ |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase-tests U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server |
| Console output | https://builds.apache.org/job/PreCommit-YARN-Build/14135/console |
| Powered by | Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
Sangjin Lee Can you link the jira that this one needs wait for? Sorry, forgot the jira number you mentioned in the meeting.
It's YARN-5739.
Uploading a rebased patch for branch YARN-5355, given that YARN-5739 has been committed to that branch. One thing that I noticed while working on YARN-5928 is that, after HBase-related code is moved into a separate module/jar, because TimelineSchemaCreator transitively depends on TimelineStorageUtil.isIntegralValue(), users will need to copy both timelineservice and timelineservice-hbase jars into hbase/lib when they create the table schemas. This is a slight inconvenience. Sangjin Lee Do you think we should keep it this way? Or we could duplicate isIntegralValue() in HBastTimelineStorageUtil to avoid copying another jar for TimelineSchemaCreator?
| +1 overall |
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| 0 | reexec | 0m 15s | Docker mode activated. |
| +1 | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 | test4tests | 0m 0s | The patch appears to include 4 new or modified test files. |
| 0 | mvndep | 3m 7s | Maven dependency ordering for branch |
| +1 | mvninstall | 8m 49s | |
| +1 | compile | 1m 37s | |
| +1 | checkstyle | 0m 31s | |
| +1 | mvnsite | 0m 46s | |
| +1 | mvneclipse | 0m 32s | |
| +1 | findbugs | 1m 4s | |
| +1 | javadoc | 0m 26s | |
| 0 | mvndep | 0m 8s | Maven dependency ordering for patch |
| +1 | mvninstall | 0m 35s | the patch passed |
| +1 | compile | 1m 42s | the patch passed |
| +1 | javac | 1m 42s | the patch passed |
| -0 | checkstyle | 0m 29s | hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server: The patch generated 1 new + 5 unchanged - 0 fixed = 6 total (was 5) |
| +1 | mvnsite | 0m 49s | the patch passed |
| +1 | mvneclipse | 0m 31s | the patch passed |
| +1 | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 | findbugs | 1m 31s | the patch passed |
| +1 | javadoc | 0m 26s | the patch passed |
| +1 | unit | 0m 49s | hadoop-yarn-server-timelineservice in the patch passed. |
| +1 | unit | 5m 33s | hadoop-yarn-server-timelineservice-hbase-tests in the patch passed. |
| +1 | asflicense | 0m 20s | The patch does not generate ASF License warnings. |
| 35m 18s |
| Subsystem | Report/Notes |
|---|---|
| Docker | Image:yetus/hadoop:9560f25 |
| JIRA Issue | |
| JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12842249/YARN-5925-YARN-5355.03.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle |
| uname | Linux 7c1ec8efed4e 3.13.0-103-generic #150-Ubuntu SMP Thu Nov 24 10:34:17 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh |
| git revision | |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle | https://builds.apache.org/job/PreCommit-YARN-Build/14215/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server.txt |
| Test Results | https://builds.apache.org/job/PreCommit-YARN-Build/14215/testReport/ |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase-tests U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server |
| Console output | https://builds.apache.org/job/PreCommit-YARN-Build/14215/console |
| Powered by | Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
I am fine with duplicating that method, as it is a pretty trivial method. However, we need to make sure there is no other dependency, direct or transitive.
That's why eventually we need to isolate the co-processor code from the rest of the HBase-related code, because the co-processor code is the only code that needs to be on the hbase server-side classpath.
cc Vrushali C
Thanks for your comments, Sangjin Lee. Attaching a new patch to duplicate it in HBaseTimelineStorageUtil and address the checkstyle warning.
LGTM. Committing shortly.
Committed it to trunk, YARN-5355, and YARN-5355-branch-2. Thanks Haibo Chen for your contribution!
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10980 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10980/)
YARN-5925. Extract hbase-backend-exclusive utility methods from (sjlee: rev 55f5886ea24671ff3db87a64aaba2e76b3355455)
- (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/main/java/org/apache/hadoop/yarn/server/timelineservice/storage/flow/FlowActivityRowKey.java
- (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/main/java/org/apache/hadoop/yarn/server/timelineservice/storage/flow/FlowRunColumn.java
- (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/test/java/org/apache/hadoop/yarn/server/timelineservice/storage/common/TestRowKeys.java
- (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/main/java/org/apache/hadoop/yarn/server/timelineservice/storage/flow/FlowActivityColumnPrefix.java
- (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/main/java/org/apache/hadoop/yarn/server/timelineservice/storage/flow/FlowRunCoprocessor.java
- (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase-tests/src/test/java/org/apache/hadoop/yarn/server/timelineservice/storage/flow/TestHBaseStorageFlowRun.java
- (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase-tests/src/test/java/org/apache/hadoop/yarn/server/timelineservice/storage/flow/TestHBaseStorageFlowRunCompaction.java
- (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/main/java/org/apache/hadoop/yarn/server/timelineservice/storage/common/AppIdKeyConverter.java
- (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/main/java/org/apache/hadoop/yarn/server/timelineservice/storage/flow/FlowRunColumnPrefix.java
- (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/main/java/org/apache/hadoop/yarn/server/timelineservice/storage/common/TimelineStorageUtils.java
- (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/main/java/org/apache/hadoop/yarn/server/timelineservice/storage/common/HBaseTimelineStorageUtils.java
- (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase-tests/src/test/java/org/apache/hadoop/yarn/server/timelineservice/reader/TestTimelineReaderWebServicesHBaseStorage.java
- (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/main/java/org/apache/hadoop/yarn/server/timelineservice/storage/flow/FlowScanner.java
- (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase-tests/src/test/java/org/apache/hadoop/yarn/server/timelineservice/storage/flow/TestHBaseStorageFlowActivity.java
Thanks Sangjin Lee for your reviews! Looks like the version you committed to trunk does not include duplication of isIntegralValue in HBaseTimelineStorageUtil. I'll add it in YARN5928 (for trunk only).
Hmm, since the trunk patch hadn't been updated, I assumed that you wanted a different version to go in for the trunk. I'm fine with making that change in YARN-5928.
Sorry for the confusion. I will make it more explicit in future jiras.
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #12271 (See https://builds.apache.org/job/Hadoop-trunk-Commit/12271/)
YARN-5925. Extract hbase-backend-exclusive utility methods from (varunsaxena: rev 2556c012e385947cfdcd23f597b174dc78efde46)
- (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase/src/main/java/org/apache/hadoop/yarn/server/timelineservice/storage/common/HBaseTimelineStorageUtils.java
- (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase/src/main/java/org/apache/hadoop/yarn/server/timelineservice/storage/HBaseTimelineWriterImpl.java
- (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase/src/main/java/org/apache/hadoop/yarn/server/timelineservice/storage/reader/EntityTypeReader.java
- (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase/src/main/java/org/apache/hadoop/yarn/server/timelineservice/storage/TimelineSchemaCreator.java
- (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase/src/main/java/org/apache/hadoop/yarn/server/timelineservice/storage/common/LongConverter.java
- (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase/src/main/java/org/apache/hadoop/yarn/server/timelineservice/storage/HBaseTimelineReaderImpl.java
YARN-5925This message was automatically generated.