Details
-
Sub-task
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
None
-
None
-
None
Description
This JIRA focuses on the refactoring of classes related to Distributed Scheduling
Attachments
Attachments
- YARN-5113.001.patch
- 125 kB
- Konstantinos Karanasos
- YARN-5113.002.patch
- 201 kB
- Konstantinos Karanasos
- YARN-5113.003.patch
- 215 kB
- Konstantinos Karanasos
- YARN-5113.004.patch
- 236 kB
- Konstantinos Karanasos
- YARN-5113.005.patch
- 152 kB
- Konstantinos Karanasos
- YARN-5113.006.patch
- 269 kB
- Konstantinos Karanasos
- YARN-5113.007.patch
- 270 kB
- Konstantinos Karanasos
- YARN-5113.008.patch
- 270 kB
- Konstantinos Karanasos
- YARN-5113.009.patch
- 276 kB
- Konstantinos Karanasos
Issue Links
- is related to
-
YARN-2885 Create AMRMProxy request interceptor for distributed scheduling decisions for queueable containers
- Resolved
Activity
-1 overall |
Vote | Subsystem | Runtime | Comment |
---|---|---|---|
0 | reexec | 0m 0s | Docker mode activated. |
-1 | patch | 0m 4s | |
Subsystem | Report/Notes |
---|---|
JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12804845/YARN-5113.001.patch |
JIRA Issue | |
Console output | https://builds.apache.org/job/PreCommit-YARN-Build/11548/console |
Powered by | Apache Yetus 0.2.0 http://yetus.apache.org |
This message was automatically generated.
Konstantinos Karanasos, the changes to MiniYarnCluster can be reverted.
-1 overall |
Vote | Subsystem | Runtime | Comment |
---|---|---|---|
0 | reexec | 0m 11s | Docker mode activated. |
+1 | @author | 0m 0s | The patch does not contain any @author tags. |
+1 | test4tests | 0m 0s | The patch appears to include 6 new or modified test files. |
0 | mvndep | 0m 20s | Maven dependency ordering for branch |
+1 | mvninstall | 6m 22s | trunk passed |
+1 | compile | 1m 53s | trunk passed |
+1 | checkstyle | 0m 40s | trunk passed |
+1 | mvnsite | 1m 56s | trunk passed |
+1 | mvneclipse | 0m 56s | trunk passed |
+1 | findbugs | 3m 17s | trunk passed |
+1 | javadoc | 1m 48s | trunk passed |
0 | mvndep | 0m 7s | Maven dependency ordering for patch |
-1 | mvninstall | 0m 11s | hadoop-yarn-server-common in the patch failed. |
-1 | mvninstall | 0m 12s | hadoop-yarn-server-nodemanager in the patch failed. |
-1 | mvninstall | 0m 14s | hadoop-yarn-server-resourcemanager in the patch failed. |
-1 | compile | 0m 38s | hadoop-yarn in the patch failed. |
-1 | cc | 0m 38s | hadoop-yarn in the patch failed. |
-1 | javac | 0m 38s | hadoop-yarn in the patch failed. |
-1 | checkstyle | 0m 36s | hadoop-yarn-project/hadoop-yarn: patch generated 12 new + 453 unchanged - 55 fixed = 465 total (was 508) |
-1 | mvnsite | 0m 12s | hadoop-yarn-server-common in the patch failed. |
-1 | mvnsite | 0m 13s | hadoop-yarn-server-nodemanager in the patch failed. |
-1 | mvnsite | 0m 15s | hadoop-yarn-server-resourcemanager in the patch failed. |
+1 | mvneclipse | 0m 45s | the patch passed |
+1 | whitespace | 0m 0s | Patch has no whitespace issues. |
+1 | xml | 0m 1s | The patch has no ill-formed XML file. |
-1 | findbugs | 0m 12s | hadoop-yarn-server-common in the patch failed. |
-1 | findbugs | 0m 12s | hadoop-yarn-server-nodemanager in the patch failed. |
-1 | findbugs | 0m 13s | hadoop-yarn-server-resourcemanager in the patch failed. |
-1 | javadoc | 2m 17s | hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common generated 118 new + 1169 unchanged - 104 fixed = 1287 total (was 1273) |
-1 | javadoc | 2m 17s | hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager generated 24 new + 585 unchanged - 7 fixed = 609 total (was 592) |
-1 | javadoc | 2m 18s | hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager generated 6 new + 1255 unchanged - 0 fixed = 1261 total (was 1255) |
+1 | javadoc | 1m 36s | the patch passed |
-1 | unit | 0m 20s | hadoop-yarn-api in the patch failed. |
-1 | unit | 0m 11s | hadoop-yarn-server-common in the patch failed. |
-1 | unit | 0m 13s | hadoop-yarn-server-nodemanager in the patch failed. |
-1 | unit | 0m 14s | hadoop-yarn-server-resourcemanager in the patch failed. |
-1 | unit | 6m 19s | hadoop-yarn-server-tests in the patch failed. |
+1 | asflicense | 0m 15s | Patch does not generate ASF License warnings. |
34m 8s |
Reason | Tests |
---|---|
Failed junit tests | hadoop.yarn.conf.TestYarnConfigurationFields |
hadoop.yarn.server.TestMiniYarnClusterNodeUtilization | |
hadoop.yarn.server.TestContainerManagerSecurity |
This message was automatically generated.
Attaching new patch (had forgotten to include some files in the last version).
-1 overall |
Vote | Subsystem | Runtime | Comment |
---|---|---|---|
0 | reexec | 0m 16s | Docker mode activated. |
+1 | @author | 0m 0s | The patch does not contain any @author tags. |
+1 | test4tests | 0m 0s | The patch appears to include 5 new or modified test files. |
0 | mvndep | 0m 18s | Maven dependency ordering for branch |
+1 | mvninstall | 6m 25s | trunk passed |
+1 | compile | 2m 4s | trunk passed |
+1 | checkstyle | 0m 43s | trunk passed |
+1 | mvnsite | 1m 46s | trunk passed |
+1 | mvneclipse | 0m 46s | trunk passed |
+1 | findbugs | 3m 12s | trunk passed |
+1 | javadoc | 1m 45s | trunk passed |
0 | mvndep | 0m 8s | Maven dependency ordering for patch |
+1 | mvninstall | 1m 30s | the patch passed |
+1 | compile | 2m 10s | the patch passed |
+1 | cc | 2m 10s | the patch passed |
+1 | javac | 2m 10s | the patch passed |
-1 | checkstyle | 0m 41s | hadoop-yarn-project/hadoop-yarn: patch generated 19 new + 403 unchanged - 44 fixed = 422 total (was 447) |
+1 | mvnsite | 1m 49s | the patch passed |
+1 | mvneclipse | 0m 39s | the patch passed |
+1 | whitespace | 0m 0s | Patch has no whitespace issues. |
+1 | xml | 0m 2s | The patch has no ill-formed XML file. |
+1 | findbugs | 3m 37s | the patch passed |
-1 | javadoc | 2m 54s | hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common generated 104 new + 1169 unchanged - 104 fixed = 1273 total (was 1273) |
-1 | javadoc | 2m 54s | hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager generated 7 new + 585 unchanged - 7 fixed = 592 total (was 592) |
+1 | javadoc | 1m 36s | the patch passed |
-1 | unit | 0m 23s | hadoop-yarn-api in the patch failed. |
+1 | unit | 0m 23s | hadoop-yarn-server-common in the patch passed. |
+1 | unit | 11m 37s | hadoop-yarn-server-nodemanager in the patch passed. |
-1 | unit | 37m 4s | hadoop-yarn-server-resourcemanager in the patch failed. |
+1 | asflicense | 0m 18s | Patch does not generate ASF License warnings. |
80m 19s |
Reason | Tests |
---|---|
Failed junit tests | hadoop.yarn.conf.TestYarnConfigurationFields |
hadoop.yarn.server.resourcemanager.TestClientRMTokens | |
hadoop.yarn.server.resourcemanager.TestAMAuthorization |
This message was automatically generated.
Konstantinos Karanasos, Looks like TestYarnConfigurationFields is a legitimate error.
Can you please run the following testcases manually and ensure they pass as well (some of them should have been triggered already, but just to be sure)
- TestQueuingContainerManager
- TestDistributedScheduling
- TestLocalScheduler (By the way, I guess this should be refactored to TestDistributedScheduler)
- TestNodeQueueLoadMonitor
Some of the javadoc errors are legit. The rest mostly relate to generated sources.
I think all the checkstyles are fixable.
+1, pending the above
Also you maybe want to update the javadoc plugin in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/pom.xml like so :
<plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-javadoc-plugin</artifactId> <configuration> <excludePackageNames>org.apache.hadoop.yarn.proto</excludePackageNames> </configuration> </plugin>
-1 overall |
Vote | Subsystem | Runtime | Comment |
---|---|---|---|
0 | reexec | 0m 13s | Docker mode activated. |
+1 | @author | 0m 0s | The patch does not contain any @author tags. |
+1 | test4tests | 0m 1s | The patch appears to include 6 new or modified test files. |
0 | mvndep | 0m 22s | Maven dependency ordering for branch |
+1 | mvninstall | 6m 10s | trunk passed |
+1 | compile | 1m 53s | trunk passed |
+1 | checkstyle | 0m 41s | trunk passed |
+1 | mvnsite | 1m 39s | trunk passed |
+1 | mvneclipse | 0m 43s | trunk passed |
+1 | findbugs | 2m 58s | trunk passed |
+1 | javadoc | 1m 4s | trunk passed |
0 | mvndep | 0m 7s | Maven dependency ordering for patch |
+1 | mvninstall | 1m 23s | the patch passed |
+1 | compile | 1m 51s | the patch passed |
-1 | cc | 1m 51s | hadoop-yarn-project_hadoop-yarn generated 1 new + 2 unchanged - 1 fixed = 3 total (was 3) |
+1 | javac | 1m 51s | the patch passed |
-1 | checkstyle | 0m 37s | hadoop-yarn-project/hadoop-yarn: The patch generated 17 new + 388 unchanged - 43 fixed = 405 total (was 431) |
+1 | mvnsite | 1m 33s | the patch passed |
+1 | mvneclipse | 0m 37s | the patch passed |
-1 | whitespace | 0m 0s | The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. |
+1 | xml | 0m 1s | The patch has no ill-formed XML file. |
+1 | findbugs | 3m 17s | the patch passed |
-1 | javadoc | 0m 12s | hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common generated 4 new + 159 unchanged - 4 fixed = 163 total (was 163) |
-1 | javadoc | 0m 14s | hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager generated 7 new + 274 unchanged - 7 fixed = 281 total (was 281) |
-1 | unit | 0m 22s | hadoop-yarn-api in the patch failed. |
+1 | unit | 0m 20s | hadoop-yarn-server-common in the patch passed. |
+1 | unit | 11m 50s | hadoop-yarn-server-nodemanager in the patch passed. |
-1 | unit | 31m 19s | hadoop-yarn-server-resourcemanager in the patch failed. |
+1 | asflicense | 0m 17s | The patch does not generate ASF License warnings. |
71m 19s |
Reason | Tests |
---|---|
Failed junit tests | hadoop.yarn.conf.TestYarnConfigurationFields |
hadoop.yarn.server.resourcemanager.TestClientRMTokens | |
hadoop.yarn.server.resourcemanager.TestRMRestart | |
hadoop.yarn.server.resourcemanager.TestAMAuthorization |
This message was automatically generated.
-1 overall |
Vote | Subsystem | Runtime | Comment |
---|---|---|---|
0 | reexec | 0m 0s | Docker mode activated. |
-1 | patch | 0m 6s | |
Subsystem | Report/Notes |
---|---|
JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12819795/YARN-5113.005.patch |
JIRA Issue | |
Console output | https://builds.apache.org/job/PreCommit-YARN-Build/12475/console |
Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
-1 overall |
Vote | Subsystem | Runtime | Comment |
---|---|---|---|
0 | reexec | 0m 17s | Docker mode activated. |
+1 | @author | 0m 0s | The patch does not contain any @author tags. |
+1 | test4tests | 0m 0s | The patch appears to include 5 new or modified test files. |
0 | mvndep | 4m 42s | Maven dependency ordering for branch |
+1 | mvninstall | 8m 5s | trunk passed |
+1 | compile | 2m 18s | trunk passed |
+1 | checkstyle | 0m 44s | trunk passed |
+1 | mvnsite | 2m 21s | trunk passed |
+1 | mvneclipse | 1m 14s | trunk passed |
+1 | findbugs | 3m 45s | trunk passed |
+1 | javadoc | 1m 23s | trunk passed |
0 | mvndep | 0m 9s | Maven dependency ordering for patch |
-1 | mvninstall | 0m 14s | hadoop-yarn-server-common in the patch failed. |
-1 | mvninstall | 0m 13s | hadoop-yarn-server-nodemanager in the patch failed. |
-1 | mvninstall | 0m 18s | hadoop-yarn-server-resourcemanager in the patch failed. |
-1 | compile | 0m 41s | hadoop-yarn in the patch failed. |
-1 | cc | 0m 41s | hadoop-yarn in the patch failed. |
-1 | javac | 0m 41s | hadoop-yarn in the patch failed. |
-1 | checkstyle | 0m 39s | hadoop-yarn-project/hadoop-yarn: The patch generated 6 new + 337 unchanged - 83 fixed = 343 total (was 420) |
-1 | mvnsite | 0m 15s | hadoop-yarn-server-common in the patch failed. |
-1 | mvnsite | 0m 14s | hadoop-yarn-server-nodemanager in the patch failed. |
-1 | mvnsite | 0m 20s | hadoop-yarn-server-resourcemanager in the patch failed. |
+1 | mvneclipse | 0m 57s | the patch passed |
+1 | whitespace | 0m 0s | The patch has no whitespace issues. |
+1 | xml | 0m 1s | The patch has no ill-formed XML file. |
-1 | findbugs | 0m 12s | hadoop-yarn-server-common in the patch failed. |
-1 | findbugs | 0m 14s | hadoop-yarn-server-nodemanager in the patch failed. |
-1 | findbugs | 0m 17s | hadoop-yarn-server-resourcemanager in the patch failed. |
+1 | javadoc | 0m 14s | hadoop-yarn-api in the patch passed. |
+1 | javadoc | 0m 12s | hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common generated 0 new + 159 unchanged - 4 fixed = 159 total (was 163) |
-1 | javadoc | 0m 14s | hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager generated 28 new + 242 unchanged - 13 fixed = 270 total (was 255) |
+1 | javadoc | 0m 18s | hadoop-yarn-server-resourcemanager in the patch passed. |
+1 | javadoc | 0m 12s | hadoop-yarn-client in the patch passed. |
-1 | unit | 0m 22s | hadoop-yarn-api in the patch failed. |
-1 | unit | 0m 15s | hadoop-yarn-server-common in the patch failed. |
-1 | unit | 0m 14s | hadoop-yarn-server-nodemanager in the patch failed. |
-1 | unit | 0m 18s | hadoop-yarn-server-resourcemanager in the patch failed. |
-1 | unit | 8m 1s | hadoop-yarn-client in the patch failed. |
+1 | asflicense | 0m 17s | The patch does not generate ASF License warnings. |
44m 11s |
Reason | Tests |
---|---|
Failed junit tests | hadoop.yarn.conf.TestYarnConfigurationFields |
hadoop.yarn.client.api.impl.TestAMRMProxy | |
hadoop.yarn.client.api.impl.TestYarnClient | |
hadoop.yarn.client.api.impl.TestDistributedScheduling |
This message was automatically generated.
-1 overall |
Vote | Subsystem | Runtime | Comment |
---|---|---|---|
0 | reexec | 0m 15s | Docker mode activated. |
+1 | @author | 0m 0s | The patch does not contain any @author tags. |
+1 | test4tests | 0m 0s | The patch appears to include 7 new or modified test files. |
0 | mvndep | 0m 9s | Maven dependency ordering for branch |
+1 | mvninstall | 6m 50s | trunk passed |
+1 | compile | 2m 22s | trunk passed |
+1 | checkstyle | 0m 44s | trunk passed |
+1 | mvnsite | 2m 20s | trunk passed |
+1 | mvneclipse | 1m 9s | trunk passed |
+1 | findbugs | 3m 48s | trunk passed |
+1 | javadoc | 1m 26s | trunk passed |
0 | mvndep | 0m 9s | Maven dependency ordering for patch |
+1 | mvninstall | 1m 55s | the patch passed |
+1 | compile | 2m 29s | the patch passed |
+1 | cc | 2m 29s | the patch passed |
+1 | javac | 2m 29s | the patch passed |
-1 | checkstyle | 0m 44s | hadoop-yarn-project/hadoop-yarn: The patch generated 85 new + 382 unchanged - 38 fixed = 467 total (was 420) |
+1 | mvnsite | 2m 10s | the patch passed |
+1 | mvneclipse | 0m 59s | the patch passed |
-1 | whitespace | 0m 0s | The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. |
+1 | xml | 0m 1s | The patch has no ill-formed XML file. |
+1 | findbugs | 4m 14s | the patch passed |
-1 | javadoc | 0m 12s | hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common generated 4 new + 159 unchanged - 4 fixed = 163 total (was 163) |
-1 | javadoc | 0m 14s | hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager generated 7 new + 248 unchanged - 7 fixed = 255 total (was 255) |
-1 | unit | 0m 22s | hadoop-yarn-api in the patch failed. |
+1 | unit | 0m 24s | hadoop-yarn-server-common in the patch passed. |
-1 | unit | 12m 55s | hadoop-yarn-server-nodemanager in the patch failed. |
+1 | unit | 33m 54s | hadoop-yarn-server-resourcemanager in the patch passed. |
-1 | unit | 9m 2s | hadoop-yarn-client in the patch failed. |
+1 | asflicense | 0m 21s | The patch does not generate ASF License warnings. |
91m 9s |
Reason | Tests |
---|---|
Failed junit tests | hadoop.yarn.conf.TestYarnConfigurationFields |
hadoop.yarn.server.nodemanager.scheduler.TestDistributedScheduler | |
hadoop.yarn.server.nodemanager.TestDirectoryCollection | |
hadoop.yarn.client.api.impl.TestDistributedScheduling | |
hadoop.yarn.client.api.impl.TestYarnClient | |
hadoop.yarn.client.api.impl.TestAMRMProxy |
This message was automatically generated.
Fixing unit test failures, checkstyle, whitespace and javadoc issues.
-1 overall |
Vote | Subsystem | Runtime | Comment |
---|---|---|---|
0 | reexec | 0m 16s | Docker mode activated. |
+1 | @author | 0m 0s | The patch does not contain any @author tags. |
+1 | test4tests | 0m 0s | The patch appears to include 7 new or modified test files. |
0 | mvndep | 5m 17s | Maven dependency ordering for branch |
+1 | mvninstall | 9m 34s | trunk passed |
+1 | compile | 2m 37s | trunk passed |
+1 | checkstyle | 0m 51s | trunk passed |
+1 | mvnsite | 3m 10s | trunk passed |
+1 | mvneclipse | 1m 31s | trunk passed |
+1 | findbugs | 5m 20s | trunk passed |
+1 | javadoc | 2m 5s | trunk passed |
0 | mvndep | 0m 9s | Maven dependency ordering for patch |
+1 | mvninstall | 2m 26s | the patch passed |
+1 | compile | 2m 46s | the patch passed |
+1 | cc | 2m 46s | the patch passed |
+1 | javac | 2m 46s | the patch passed |
-1 | checkstyle | 0m 47s | hadoop-yarn-project/hadoop-yarn: The patch generated 7 new + 367 unchanged - 45 fixed = 374 total (was 412) |
+1 | mvnsite | 2m 58s | the patch passed |
+1 | mvneclipse | 1m 19s | the patch passed |
+1 | whitespace | 0m 0s | The patch has no whitespace issues. |
+1 | xml | 0m 3s | The patch has no ill-formed XML file. |
+1 | findbugs | 6m 10s | the patch passed |
+1 | javadoc | 0m 16s | hadoop-yarn-api in the patch passed. |
+1 | javadoc | 0m 28s | hadoop-yarn-common in the patch passed. |
+1 | javadoc | 0m 13s | hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common generated 0 new + 159 unchanged - 4 fixed = 159 total (was 163) |
+1 | javadoc | 0m 16s | hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager generated 0 new + 248 unchanged - 7 fixed = 248 total (was 255) |
+1 | javadoc | 0m 19s | hadoop-yarn-server-resourcemanager in the patch passed. |
+1 | javadoc | 0m 13s | hadoop-yarn-client in the patch passed. |
+1 | unit | 0m 28s | hadoop-yarn-api in the patch passed. |
+1 | unit | 2m 31s | hadoop-yarn-common in the patch passed. |
+1 | unit | 0m 29s | hadoop-yarn-server-common in the patch passed. |
-1 | unit | 13m 22s | hadoop-yarn-server-nodemanager in the patch failed. |
-1 | unit | 34m 40s | hadoop-yarn-server-resourcemanager in the patch failed. |
-1 | unit | 8m 6s | hadoop-yarn-client in the patch failed. |
+1 | asflicense | 0m 18s | The patch does not generate ASF License warnings. |
110m 26s |
Reason | Tests |
---|---|
Failed junit tests | hadoop.yarn.server.nodemanager.TestDirectoryCollection |
hadoop.yarn.server.resourcemanager.scheduler.fair.TestContinuousScheduling | |
hadoop.yarn.client.api.impl.TestYarnClient | |
hadoop.yarn.client.api.impl.TestAMRMProxy | |
hadoop.yarn.client.api.impl.TestDistributedScheduling |
This message was automatically generated.
Fixed one remaining checkstyle issue.
The failing test cases are unrelated and have been fixed.
There is only one still failing (at least locally), namely TestDirectoryCollection::testCreateDirectories, which is also failing in trunk.
-1 overall |
Vote | Subsystem | Runtime | Comment |
---|---|---|---|
0 | reexec | 0m 10s | Docker mode activated. |
+1 | @author | 0m 0s | The patch does not contain any @author tags. |
+1 | test4tests | 0m 0s | The patch appears to include 7 new or modified test files. |
0 | mvndep | 0m 9s | Maven dependency ordering for branch |
+1 | mvninstall | 8m 16s | trunk passed |
+1 | compile | 2m 57s | trunk passed |
+1 | checkstyle | 0m 45s | trunk passed |
+1 | mvnsite | 3m 10s | trunk passed |
+1 | mvneclipse | 1m 29s | trunk passed |
+1 | findbugs | 5m 26s | trunk passed |
+1 | javadoc | 1m 55s | trunk passed |
0 | mvndep | 0m 11s | Maven dependency ordering for patch |
+1 | mvninstall | 2m 42s | the patch passed |
+1 | compile | 2m 39s | the patch passed |
+1 | cc | 2m 39s | the patch passed |
+1 | javac | 2m 39s | the patch passed |
-1 | checkstyle | 0m 40s | hadoop-yarn-project/hadoop-yarn: The patch generated 6 new + 367 unchanged - 45 fixed = 373 total (was 412) |
+1 | mvnsite | 2m 43s | the patch passed |
+1 | mvneclipse | 1m 14s | the patch passed |
+1 | whitespace | 0m 0s | The patch has no whitespace issues. |
+1 | xml | 0m 2s | The patch has no ill-formed XML file. |
+1 | findbugs | 6m 16s | the patch passed |
+1 | javadoc | 0m 15s | hadoop-yarn-api in the patch passed. |
+1 | javadoc | 0m 26s | hadoop-yarn-common in the patch passed. |
+1 | javadoc | 0m 14s | hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common generated 0 new + 159 unchanged - 4 fixed = 159 total (was 163) |
+1 | javadoc | 0m 15s | hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager generated 0 new + 248 unchanged - 7 fixed = 248 total (was 255) |
+1 | javadoc | 0m 19s | hadoop-yarn-server-resourcemanager in the patch passed. |
+1 | javadoc | 0m 12s | hadoop-yarn-client in the patch passed. |
+1 | unit | 0m 23s | hadoop-yarn-api in the patch passed. |
+1 | unit | 2m 15s | hadoop-yarn-common in the patch passed. |
+1 | unit | 0m 25s | hadoop-yarn-server-common in the patch passed. |
-1 | unit | 13m 4s | hadoop-yarn-server-nodemanager in the patch failed. |
-1 | unit | 33m 23s | hadoop-yarn-server-resourcemanager in the patch failed. |
-1 | unit | 8m 20s | hadoop-yarn-client in the patch failed. |
+1 | asflicense | 0m 18s | The patch does not generate ASF License warnings. |
102m 0s |
Reason | Tests |
---|---|
Failed junit tests | hadoop.yarn.server.nodemanager.TestDirectoryCollection |
hadoop.yarn.server.resourcemanager.scheduler.fair.TestContinuousScheduling | |
hadoop.yarn.client.api.impl.TestYarnClient | |
hadoop.yarn.client.api.impl.TestAMRMProxy | |
hadoop.yarn.client.api.impl.TestDistributedScheduling |
This message was automatically generated.
Most failing tests are unrelated.
The only related one is TestDistributedScheduling, which however runs successfully locally.
Thanks Konstantinos Karanasos, Couple of nits:
- Since we are refactoring, lets keep the configurations consistent, ..distributed-scheduling.enable instead of ..enabled (I see both kinds in YarnConfiguration, but I see more 'enable' than 'enabled')
- Also, currently, most of the distributed scheduling config field checking are ignored in TestYarnConfiguration, lets not ignore it... This means we must also include the default values in yarn-defaults.xml
- Can we add comments for all the Request and Response classes ?
- Minor nit: In DistributedSchedulingAMProtocol, lets declare the allocate method before the finish method.
Thanks for the comments, Arun Suresh. Uploading new patch.
I added all the distributed/NM-queuing related parameters to yarn-default.xml and got rid of skipping those variables in TestYarnConfigurationFields.
I also added/updated the comments for the Request and Reponse classes, and moved the finish method after the allocate one in the distributed_scheduling_am_protocol.
As Arun points out, both "enable" and "enabled" are used in the YarnConfiguration for various parameters. I checked again and kept the "enabled" for distributed scheduling, because it is used more times than the "enable" (~40 and 10 times, respectively).
-1 overall |
Vote | Subsystem | Runtime | Comment |
---|---|---|---|
0 | reexec | 0m 17s | Docker mode activated. |
+1 | @author | 0m 0s | The patch does not contain any @author tags. |
+1 | test4tests | 0m 0s | The patch appears to include 7 new or modified test files. |
0 | mvndep | 1m 21s | Maven dependency ordering for branch |
+1 | mvninstall | 6m 41s | trunk passed |
+1 | compile | 2m 19s | trunk passed |
+1 | checkstyle | 0m 42s | trunk passed |
+1 | mvnsite | 2m 48s | trunk passed |
+1 | mvneclipse | 1m 27s | trunk passed |
+1 | findbugs | 4m 33s | trunk passed |
+1 | javadoc | 1m 54s | trunk passed |
0 | mvndep | 0m 9s | Maven dependency ordering for patch |
+1 | mvninstall | 2m 16s | the patch passed |
+1 | compile | 2m 14s | the patch passed |
+1 | cc | 2m 14s | the patch passed |
+1 | javac | 2m 14s | the patch passed |
-1 | checkstyle | 0m 40s | hadoop-yarn-project/hadoop-yarn: The patch generated 7 new + 363 unchanged - 47 fixed = 370 total (was 410) |
+1 | mvnsite | 2m 35s | the patch passed |
+1 | mvneclipse | 1m 12s | the patch passed |
+1 | whitespace | 0m 0s | The patch has no whitespace issues. |
+1 | xml | 0m 2s | The patch has no ill-formed XML file. |
+1 | findbugs | 5m 10s | the patch passed |
+1 | javadoc | 0m 16s | hadoop-yarn-api in the patch passed. |
+1 | javadoc | 0m 25s | hadoop-yarn-common in the patch passed. |
+1 | javadoc | 0m 13s | hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common generated 0 new + 159 unchanged - 4 fixed = 159 total (was 163) |
+1 | javadoc | 0m 14s | hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager generated 0 new + 248 unchanged - 7 fixed = 248 total (was 255) |
+1 | javadoc | 0m 18s | hadoop-yarn-server-resourcemanager in the patch passed. |
+1 | javadoc | 0m 12s | hadoop-yarn-client in the patch passed. |
+1 | unit | 0m 23s | hadoop-yarn-api in the patch passed. |
+1 | unit | 2m 14s | hadoop-yarn-common in the patch passed. |
+1 | unit | 0m 25s | hadoop-yarn-server-common in the patch passed. |
-1 | unit | 13m 14s | hadoop-yarn-server-nodemanager in the patch failed. |
+1 | unit | 36m 16s | hadoop-yarn-server-resourcemanager in the patch passed. |
-1 | unit | 8m 16s | hadoop-yarn-client in the patch failed. |
+1 | asflicense | 0m 17s | The patch does not generate ASF License warnings. |
100m 23s |
Reason | Tests |
---|---|
Failed junit tests | hadoop.yarn.server.nodemanager.TestDirectoryCollection |
hadoop.yarn.client.api.impl.TestAMRMProxy | |
hadoop.yarn.client.api.impl.TestYarnClient | |
hadoop.yarn.client.api.impl.TestDistributedScheduling |
Subsystem | Report/Notes |
---|---|
Docker | Image:yetus/hadoop:9560f25 |
JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12821180/YARN-5113.009.patch |
JIRA Issue | |
Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml cc |
uname | Linux 7f0c60ca90b9 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
Build tool | maven |
Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh |
git revision | trunk / ef501b1 |
Default Java | 1.8.0_101 |
findbugs | v3.0.0 |
checkstyle | https://builds.apache.org/job/PreCommit-YARN-Build/12575/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt |
unit | https://builds.apache.org/job/PreCommit-YARN-Build/12575/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt |
unit | https://builds.apache.org/job/PreCommit-YARN-Build/12575/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt |
unit test logs | https://builds.apache.org/job/PreCommit-YARN-Build/12575/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt https://builds.apache.org/job/PreCommit-YARN-Build/12575/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt |
Test Results | https://builds.apache.org/job/PreCommit-YARN-Build/12575/testReport/ |
modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client U: hadoop-yarn-project/hadoop-yarn |
Console output | https://builds.apache.org/job/PreCommit-YARN-Build/12575/console |
Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
+1, Committing this shortly (will fix the indentation check-style issue when I check in)
There are still further opportunities to refactor, especially in the context of YARN-5220, since a lot of the code can be re-used for centralized placement. Shall open a separate JIRA to track it.
SUCCESS: Integrated in Hadoop-trunk-Commit #10184 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10184/)
YARN-5113. Refactoring and other clean-up for distributed scheduling. (arun suresh: rev e5766b1dbee02ae0ef89618e172f3fb227af19e8)
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/api/DistributedSchedulingAMProtocol.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/api/protocolrecords/DistSchedAllocateResponse.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/api/protocolrecords/impl/pb/DistributedSchedulingAllocateRequestPBImpl.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/proto/distributed_scheduling_am_protocol.proto
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/api/impl/TestDistributedScheduling.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/amrmproxy/RequestInterceptor.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/amrmproxy/DefaultRequestInterceptor.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestDistributedSchedulingAMService.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/api/ServerRMProxy.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/api/protocolrecords/impl/pb/DistSchedAllocateResponsePBImpl.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/amrmproxy/AbstractRequestInterceptor.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/scheduler/TestLocalScheduler.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/test/java/org/apache/hadoop/yarn/conf/TestYarnConfigurationFields.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/api/protocolrecords/impl/pb/DistributedSchedulingAllocateResponsePBImpl.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/scheduler/OpportunisticContainerAllocator.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/api/impl/pb/client/DistributedSchedulerProtocolPBClientImpl.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/api/protocolrecords/DistSchedRegisterResponse.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/api/DistributedSchedulerProtocol.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/api/protocolrecords/impl/pb/DistSchedRegisterResponsePBImpl.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/pom.xml
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/amrmproxy/AMRMProxyService.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/scheduler/DistributedScheduler.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/proto/distributed_scheduler_protocol.proto
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/api/DistributedSchedulingAMProtocolPB.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/api/protocolrecords/DistributedSchedulingAllocateRequest.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/DistributedSchedulingService.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/api/impl/pb/service/DistributedSchedulingAMProtocolPBServiceImpl.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/api/protocolrecords/DistSchedAllocateRequest.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/api/protocolrecords/DistributedSchedulingAllocateResponse.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestDistributedSchedulingService.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/api/DistributedSchedulerProtocolPB.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/scheduler/TestDistributedScheduler.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/scheduler/LocalScheduler.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/proto/yarn_server_common_service_protos.proto
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/AppSchedulingInfo.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/api/protocolrecords/RegisterDistributedSchedulingAMResponse.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/api/protocolrecords/impl/pb/DistSchedAllocateRequestPBImpl.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/api/protocolrecords/impl/pb/RegisterDistributedSchedulingAMResponsePBImpl.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/api/impl/pb/client/DistributedSchedulingAMProtocolPBClientImpl.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockRM.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/api/impl/pb/service/DistributedSchedulerProtocolPBServiceImpl.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/DistributedSchedulingAMService.java
Adding first version of the patch.