Details
-
Sub-task
-
Status: Resolved
-
Blocker
-
Resolution: Fixed
-
None
-
None
-
None
-
Reviewed
Description
Our javadocs only show annotated and @Public APIs (original JIRAs HADOOP-7782, HADOOP-6658).
But the jdiff shows all APIs that are not annotated @Private. This causes confusion on how we read the reports and what APIs we really broke.
Attachments
Attachments
- HADOOP-13544-20160825.txt
- 4.56 MB
- Vinod Kumar Vavilapalli
- HADOOP-13544-20160921.txt
- 3.91 MB
- Wangda Tan
Activity
Hadoop QA
added a comment -
-1 overall |
Vote | Subsystem | Runtime | Comment |
---|---|---|---|
0 | reexec | 0m 25s | Docker mode activated. |
0 | patch | 0m 1s | The patch file was not named according to hadoop's naming conventions. Please see https://wiki.apache.org/hadoop/HowToContribute for instructions. |
+1 | @author | 0m 0s | The patch does not contain any @author tags. |
-1 | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. |
0 | mvndep | 0m 33s | Maven dependency ordering for branch |
+1 | mvninstall | 6m 36s | trunk passed |
+1 | compile | 6m 52s | trunk passed |
+1 | checkstyle | 1m 23s | trunk passed |
+1 | mvnsite | 8m 27s | trunk passed |
+1 | mvneclipse | 2m 6s | trunk passed |
0 | findbugs | 0m 0s | Skipped patched modules with no Java source: hadoop-project-dist hadoop-yarn-project/hadoop-yarn hadoop-mapreduce-project/hadoop-mapreduce-client hadoop-mapreduce-project |
+1 | findbugs | 3m 21s | trunk passed |
+1 | javadoc | 5m 23s | trunk passed |
0 | mvndep | 0m 16s | Maven dependency ordering for patch |
+1 | mvninstall | 6m 51s | the patch passed |
+1 | compile | 6m 50s | the patch passed |
+1 | javac | 6m 50s | the patch passed |
-0 | checkstyle | 1m 26s | root: The patch generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0) |
+1 | mvnsite | 8m 46s | the patch passed |
+1 | mvneclipse | 2m 29s | the patch passed |
-1 | whitespace | 0m 0s | The patch 3 line(s) with tabs. |
+1 | xml | 0m 15s | The patch has no ill-formed XML file. |
0 | findbugs | 0m 0s | Skipped patched modules with no Java source: hadoop-project-dist hadoop-yarn-project/hadoop-yarn hadoop-mapreduce-project/hadoop-mapreduce-client hadoop-mapreduce-project |
+1 | findbugs | 3m 53s | the patch passed |
+1 | javadoc | 5m 47s | the patch passed |
+1 | unit | 0m 16s | hadoop-annotations in the patch passed. |
+1 | unit | 0m 15s | hadoop-project-dist in the patch passed. |
+1 | unit | 8m 12s | hadoop-common in the patch passed. |
-1 | unit | 57m 52s | hadoop-hdfs in the patch failed. |
-1 | unit | 19m 0s | hadoop-yarn in the patch failed. |
-1 | unit | 149m 40s | hadoop-mapreduce-client in the patch failed. |
-1 | unit | 150m 7s | hadoop-mapreduce-project in the patch failed. |
-1 | asflicense | 0m 41s | The patch generated 3 ASF License warnings. |
482m 15s |
Reason | Tests |
---|---|
Failed junit tests | hadoop.hdfs.TestEncryptionZones |
hadoop.yarn.server.applicationhistoryservice.webapp.TestAHSWebServices |
This message was automatically generated.
Hadoop QA
added a comment -
-1 overall
Vote
Subsystem
Runtime
Comment
0
reexec
0m 25s
Docker mode activated.
0
patch
0m 1s
The patch file was not named according to hadoop's naming conventions. Please see https://wiki.apache.org/hadoop/HowToContribute for instructions.
+1
@author
0m 0s
The patch does not contain any @author tags.
-1
test4tests
0m 0s
The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
0
mvndep
0m 33s
Maven dependency ordering for branch
+1
mvninstall
6m 36s
trunk passed
+1
compile
6m 52s
trunk passed
+1
checkstyle
1m 23s
trunk passed
+1
mvnsite
8m 27s
trunk passed
+1
mvneclipse
2m 6s
trunk passed
0
findbugs
0m 0s
Skipped patched modules with no Java source: hadoop-project-dist hadoop-yarn-project/hadoop-yarn hadoop-mapreduce-project/hadoop-mapreduce-client hadoop-mapreduce-project
+1
findbugs
3m 21s
trunk passed
+1
javadoc
5m 23s
trunk passed
0
mvndep
0m 16s
Maven dependency ordering for patch
+1
mvninstall
6m 51s
the patch passed
+1
compile
6m 50s
the patch passed
+1
javac
6m 50s
the patch passed
-0
checkstyle
1m 26s
root: The patch generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0)
+1
mvnsite
8m 46s
the patch passed
+1
mvneclipse
2m 29s
the patch passed
-1
whitespace
0m 0s
The patch 3 line(s) with tabs.
+1
xml
0m 15s
The patch has no ill-formed XML file.
0
findbugs
0m 0s
Skipped patched modules with no Java source: hadoop-project-dist hadoop-yarn-project/hadoop-yarn hadoop-mapreduce-project/hadoop-mapreduce-client hadoop-mapreduce-project
+1
findbugs
3m 53s
the patch passed
+1
javadoc
5m 47s
the patch passed
+1
unit
0m 16s
hadoop-annotations in the patch passed.
+1
unit
0m 15s
hadoop-project-dist in the patch passed.
+1
unit
8m 12s
hadoop-common in the patch passed.
-1
unit
57m 52s
hadoop-hdfs in the patch failed.
-1
unit
19m 0s
hadoop-yarn in the patch failed.
-1
unit
149m 40s
hadoop-mapreduce-client in the patch failed.
-1
unit
150m 7s
hadoop-mapreduce-project in the patch failed.
-1
asflicense
0m 41s
The patch generated 3 ASF License warnings.
482m 15s
Reason
Tests
Failed junit tests
hadoop.hdfs.TestEncryptionZones
hadoop.yarn.server.applicationhistoryservice.webapp.TestAHSWebServices
Subsystem
Report/Notes
Docker
Image:yetus/hadoop:9560f25
JIRA Issue
HADOOP-13544
JIRA Patch URL
https://issues.apache.org/jira/secure/attachment/12825563/HADOOP-13544-20160825.txt
Optional Tests
asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
uname
Linux d69c17afa069 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
Build tool
maven
Personality
/testptch/hadoop/patchprocess/precommit/personality/provided.sh
git revision
trunk / 81485db
Default Java
1.8.0_101
findbugs
v3.0.0
checkstyle
https://builds.apache.org/job/PreCommit-HADOOP-Build/10378/artifact/patchprocess/diff-checkstyle-root.txt
whitespace
https://builds.apache.org/job/PreCommit-HADOOP-Build/10378/artifact/patchprocess/whitespace-tabs.txt
unit
https://builds.apache.org/job/PreCommit-HADOOP-Build/10378/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
unit
https://builds.apache.org/job/PreCommit-HADOOP-Build/10378/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn.txt
unit
https://builds.apache.org/job/PreCommit-HADOOP-Build/10378/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client.txt
unit
https://builds.apache.org/job/PreCommit-HADOOP-Build/10378/artifact/patchprocess/patch-unit-hadoop-mapreduce-project.txt
Test Results
https://builds.apache.org/job/PreCommit-HADOOP-Build/10378/testReport/
asflicense
https://builds.apache.org/job/PreCommit-HADOOP-Build/10378/artifact/patchprocess/patch-asflicense-problems.txt
modules
C: hadoop-common-project/hadoop-annotations hadoop-project-dist hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs hadoop-yarn-project/hadoop-yarn hadoop-mapreduce-project/hadoop-mapreduce-client hadoop-mapreduce-project U: .
Console output
https://builds.apache.org/job/PreCommit-HADOOP-Build/10378/console
Powered by
Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org
This message was automatically generated.
Wangda Tan
added a comment -
Thanks for working on this issue, could you update Apache license header or exclude from rat plugin for following files?
- hadoop-mapreduce-project/dev-support/jdiff/Apache_Hadoop_MapReduce_JobClient_2.7.2.xml
- hadoop-mapreduce-project/dev-support/jdiff/Apache_Hadoop_MapReduce_Core_2.7.2.xml
- hadoop-mapreduce-project/dev-support/jdiff/Apache_Hadoop_MapReduce_Common_2.7.2.xml
Beyond this, patch looks good to me.
Wangda Tan
added a comment - vinodkv ,
Thanks for working on this issue, could you update Apache license header or exclude from rat plugin for following files?
hadoop-mapreduce-project/dev-support/jdiff/Apache_Hadoop_MapReduce_JobClient_2.7.2.xml
hadoop-mapreduce-project/dev-support/jdiff/Apache_Hadoop_MapReduce_Core_2.7.2.xml
hadoop-mapreduce-project/dev-support/jdiff/Apache_Hadoop_MapReduce_Common_2.7.2.xml
Beyond this, patch looks good to me.
Wangda Tan
added a comment - Fixed ASF license issues.
Hadoop QA
added a comment -
-1 overall |
Vote | Subsystem | Runtime | Comment |
---|---|---|---|
0 | reexec | 0m 13s | Docker mode activated. |
0 | patch | 0m 1s | The patch file was not named according to hadoop's naming conventions. Please see https://wiki.apache.org/hadoop/HowToContribute for instructions. |
+1 | @author | 0m 0s | The patch does not contain any @author tags. |
-1 | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. |
0 | mvndep | 0m 38s | Maven dependency ordering for branch |
+1 | mvninstall | 7m 15s | trunk passed |
+1 | compile | 7m 11s | trunk passed |
+1 | checkstyle | 1m 29s | trunk passed |
+1 | mvnsite | 8m 45s | trunk passed |
+1 | mvneclipse | 2m 11s | trunk passed |
0 | findbugs | 0m 0s | Skipped patched modules with no Java source: hadoop-project-dist hadoop-yarn-project/hadoop-yarn hadoop-mapreduce-project/hadoop-mapreduce-client hadoop-mapreduce-project |
+1 | findbugs | 3m 32s | trunk passed |
+1 | javadoc | 5m 35s | trunk passed |
0 | mvndep | 0m 17s | Maven dependency ordering for patch |
+1 | mvninstall | 7m 17s | the patch passed |
+1 | compile | 7m 24s | the patch passed |
+1 | javac | 7m 24s | the patch passed |
-0 | checkstyle | 1m 33s | root: The patch generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0) |
+1 | mvnsite | 9m 25s | the patch passed |
+1 | mvneclipse | 2m 29s | the patch passed |
-1 | whitespace | 0m 0s | The patch 3 line(s) with tabs. |
+1 | xml | 0m 15s | The patch has no ill-formed XML file. |
0 | findbugs | 0m 0s | Skipped patched modules with no Java source: hadoop-project-dist hadoop-yarn-project/hadoop-yarn hadoop-mapreduce-project/hadoop-mapreduce-client hadoop-mapreduce-project |
+1 | findbugs | 4m 8s | the patch passed |
+1 | javadoc | 6m 0s | the patch passed |
+1 | unit | 0m 15s | hadoop-annotations in the patch passed. |
+1 | unit | 0m 14s | hadoop-project-dist in the patch passed. |
+1 | unit | 8m 15s | hadoop-common in the patch passed. |
-1 | unit | 62m 29s | hadoop-hdfs in the patch failed. |
-1 | unit | 21m 6s | hadoop-yarn in the patch failed. |
-1 | unit | 146m 58s | hadoop-mapreduce-client in the patch failed. |
-1 | unit | 147m 41s | hadoop-mapreduce-project in the patch failed. |
+1 | asflicense | 0m 39s | The patch does not generate ASF License warnings. |
487m 21s |
Reason | Tests |
---|---|
Failed junit tests | hadoop.hdfs.TestMaintenanceState |
hadoop.yarn.server.applicationhistoryservice.webapp.TestAHSWebServices |
This message was automatically generated.
Hadoop QA
added a comment -
-1 overall
Vote
Subsystem
Runtime
Comment
0
reexec
0m 13s
Docker mode activated.
0
patch
0m 1s
The patch file was not named according to hadoop's naming conventions. Please see https://wiki.apache.org/hadoop/HowToContribute for instructions.
+1
@author
0m 0s
The patch does not contain any @author tags.
-1
test4tests
0m 0s
The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
0
mvndep
0m 38s
Maven dependency ordering for branch
+1
mvninstall
7m 15s
trunk passed
+1
compile
7m 11s
trunk passed
+1
checkstyle
1m 29s
trunk passed
+1
mvnsite
8m 45s
trunk passed
+1
mvneclipse
2m 11s
trunk passed
0
findbugs
0m 0s
Skipped patched modules with no Java source: hadoop-project-dist hadoop-yarn-project/hadoop-yarn hadoop-mapreduce-project/hadoop-mapreduce-client hadoop-mapreduce-project
+1
findbugs
3m 32s
trunk passed
+1
javadoc
5m 35s
trunk passed
0
mvndep
0m 17s
Maven dependency ordering for patch
+1
mvninstall
7m 17s
the patch passed
+1
compile
7m 24s
the patch passed
+1
javac
7m 24s
the patch passed
-0
checkstyle
1m 33s
root: The patch generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0)
+1
mvnsite
9m 25s
the patch passed
+1
mvneclipse
2m 29s
the patch passed
-1
whitespace
0m 0s
The patch 3 line(s) with tabs.
+1
xml
0m 15s
The patch has no ill-formed XML file.
0
findbugs
0m 0s
Skipped patched modules with no Java source: hadoop-project-dist hadoop-yarn-project/hadoop-yarn hadoop-mapreduce-project/hadoop-mapreduce-client hadoop-mapreduce-project
+1
findbugs
4m 8s
the patch passed
+1
javadoc
6m 0s
the patch passed
+1
unit
0m 15s
hadoop-annotations in the patch passed.
+1
unit
0m 14s
hadoop-project-dist in the patch passed.
+1
unit
8m 15s
hadoop-common in the patch passed.
-1
unit
62m 29s
hadoop-hdfs in the patch failed.
-1
unit
21m 6s
hadoop-yarn in the patch failed.
-1
unit
146m 58s
hadoop-mapreduce-client in the patch failed.
-1
unit
147m 41s
hadoop-mapreduce-project in the patch failed.
+1
asflicense
0m 39s
The patch does not generate ASF License warnings.
487m 21s
Reason
Tests
Failed junit tests
hadoop.hdfs.TestMaintenanceState
hadoop.yarn.server.applicationhistoryservice.webapp.TestAHSWebServices
Subsystem
Report/Notes
Docker
Image:yetus/hadoop:9560f25
JIRA Issue
HADOOP-13544
JIRA Patch URL
https://issues.apache.org/jira/secure/attachment/12829725/HADOOP-13544-20160921.txt
Optional Tests
asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
uname
Linux fe730630f674 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
Build tool
maven
Personality
/testptch/hadoop/patchprocess/precommit/personality/provided.sh
git revision
trunk / 964e546
Default Java
1.8.0_101
findbugs
v3.0.0
checkstyle
https://builds.apache.org/job/PreCommit-HADOOP-Build/10563/artifact/patchprocess/diff-checkstyle-root.txt
whitespace
https://builds.apache.org/job/PreCommit-HADOOP-Build/10563/artifact/patchprocess/whitespace-tabs.txt
unit
https://builds.apache.org/job/PreCommit-HADOOP-Build/10563/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
unit
https://builds.apache.org/job/PreCommit-HADOOP-Build/10563/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn.txt
unit
https://builds.apache.org/job/PreCommit-HADOOP-Build/10563/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client.txt
unit
https://builds.apache.org/job/PreCommit-HADOOP-Build/10563/artifact/patchprocess/patch-unit-hadoop-mapreduce-project.txt
Test Results
https://builds.apache.org/job/PreCommit-HADOOP-Build/10563/testReport/
modules
C: hadoop-common-project/hadoop-annotations hadoop-project-dist hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs hadoop-yarn-project/hadoop-yarn hadoop-mapreduce-project/hadoop-mapreduce-client hadoop-mapreduce-project U: .
Console output
https://builds.apache.org/job/PreCommit-HADOOP-Build/10563/console
Powered by
Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org
This message was automatically generated.
Wangda Tan
added a comment - +1 for the latest patch, test failure is not related.
Wangda Tan
added a comment - Committed to branch-2/2.8/trunk, thanks vinodkv
Hudson
added a comment -
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10499 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10499/)
HADOOP-13544. JDiff reports unncessarily show unannotated APIs and cause (wangda: rev 875062b5bc789158290bf93dadc71b5328ca4fee)
- (edit) hadoop-project-dist/pom.xml
- (add) hadoop-common-project/hadoop-annotations/src/main/java/org/apache/hadoop/classification/tools/IncludePublicAnnotationsJDiffDoclet.java
- (edit) hadoop-mapreduce-project/dev-support/jdiff/Apache_Hadoop_MapReduce_JobClient_2.7.2.xml
- (edit) hadoop-yarn-project/hadoop-yarn/dev-support/jdiff/Apache_Hadoop_YARN_Server_Common_2.7.2.xml
- (edit) hadoop-yarn-project/hadoop-yarn/dev-support/jdiff/Apache_Hadoop_YARN_Client_2.7.2.xml
- (edit) hadoop-yarn-project/hadoop-yarn/pom.xml
- (edit) hadoop-yarn-project/hadoop-yarn/dev-support/jdiff/Apache_Hadoop_YARN_API_2.7.2.xml
- (edit) hadoop-common-project/hadoop-common/dev-support/jdiff/Apache_Hadoop_Common_2.7.2.xml
- (edit) hadoop-mapreduce-project/dev-support/jdiff/Apache_Hadoop_MapReduce_Core_2.7.2.xml
- (edit) hadoop-yarn-project/hadoop-yarn/dev-support/jdiff/Apache_Hadoop_YARN_Common_2.7.2.xml
- (edit) hadoop-mapreduce-project/dev-support/jdiff/Apache_Hadoop_MapReduce_Common_2.7.2.xml
- (edit) hadoop-hdfs-project/hadoop-hdfs/dev-support/jdiff/Apache_Hadoop_HDFS_2.7.2.xml
- (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/pom.xml
Hudson
added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10499 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10499/ )
HADOOP-13544 . JDiff reports unncessarily show unannotated APIs and cause (wangda: rev 875062b5bc789158290bf93dadc71b5328ca4fee)
(edit) hadoop-project-dist/pom.xml
(add) hadoop-common-project/hadoop-annotations/src/main/java/org/apache/hadoop/classification/tools/IncludePublicAnnotationsJDiffDoclet.java
(edit) hadoop-mapreduce-project/dev-support/jdiff/Apache_Hadoop_MapReduce_JobClient_2.7.2.xml
(edit) hadoop-yarn-project/hadoop-yarn/dev-support/jdiff/Apache_Hadoop_YARN_Server_Common_2.7.2.xml
(edit) hadoop-yarn-project/hadoop-yarn/dev-support/jdiff/Apache_Hadoop_YARN_Client_2.7.2.xml
(edit) hadoop-yarn-project/hadoop-yarn/pom.xml
(edit) hadoop-yarn-project/hadoop-yarn/dev-support/jdiff/Apache_Hadoop_YARN_API_2.7.2.xml
(edit) hadoop-common-project/hadoop-common/dev-support/jdiff/Apache_Hadoop_Common_2.7.2.xml
(edit) hadoop-mapreduce-project/dev-support/jdiff/Apache_Hadoop_MapReduce_Core_2.7.2.xml
(edit) hadoop-yarn-project/hadoop-yarn/dev-support/jdiff/Apache_Hadoop_YARN_Common_2.7.2.xml
(edit) hadoop-mapreduce-project/dev-support/jdiff/Apache_Hadoop_MapReduce_Common_2.7.2.xml
(edit) hadoop-hdfs-project/hadoop-hdfs/dev-support/jdiff/Apache_Hadoop_HDFS_2.7.2.xml
(edit) hadoop-mapreduce-project/hadoop-mapreduce-client/pom.xml
Here's a patch that fixes the problem. Now, just like our official javadocs, JDiff report only looks at annotated APIs that are marked Public.
The patch is large because I also went back to the old 2.7.2 release and regenerated the jdiff files following this new convention.