Details
-
Type:
Bug
-
Status: Resolved
-
Priority:
Major
-
Resolution: Fixed
-
Affects Version/s: 3.0.0-alpha1
-
Fix Version/s: 3.0.0-alpha1
-
Component/s: timelineserver
-
Labels:None
-
Hadoop Flags:Reviewed
Description
TestDistributedShell.checkTimelineV2() uses the default (hard-coded) storage root directory. This is brittle against concurrent tests. We should use a unique storage directory for the unit tests.
Issue Links
- is depended upon by
-
YARN-5359
FileSystemTimelineReader/Writer uses unix-specific default storage path
-
- Resolved
-
- is required by
-
MAPREDUCE-6731
TestMRTimelineEventHandling.testMRNewTimelineServiceEventHandling() may fail for concurrent tests
-
- Resolved
-
Activity
- All
- Comments
- Work Log
- History
- Activity
- Transitions
 +1 overall |
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| 0 | reexec | 0m 15s | Docker mode activated. |
| +1 | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. |
| +1 | mvninstall | 6m 59s | trunk passed |
| +1 | compile | 0m 18s | trunk passed |
| +1 | checkstyle | 0m 13s | trunk passed |
| +1 | mvnsite | 0m 23s | trunk passed |
| +1 | mvneclipse | 0m 17s | trunk passed |
| +1 | findbugs | 0m 25s | trunk passed |
| +1 | javadoc | 0m 13s | trunk passed |
| +1 | mvninstall | 0m 16s | the patch passed |
| +1 | compile | 0m 15s | the patch passed |
| +1 | javac | 0m 15s | the patch passed |
| +1 | checkstyle | 0m 11s | hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell: The patch generated 0 new + 78 unchanged - 1 fixed = 78 total (was 79) |
| +1 | mvnsite | 0m 19s | the patch passed |
| +1 | mvneclipse | 0m 13s | the patch passed |
| +1 | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 | findbugs | 0m 31s | the patch passed |
| +1 | javadoc | 0m 10s | the patch passed |
| +1 | unit | 9m 31s | hadoop-yarn-applications-distributedshell in the patch passed. |
| +1 | asflicense | 0m 16s | The patch does not generate ASF License warnings. |
| 21m 21s |
| Subsystem | Report/Notes |
|---|---|
| Docker | Image:yetus/hadoop:9560f25 |
| JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12817263/YARN-5354.01.patch |
| JIRA Issue | |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle |
| uname | Linux f5ae9b3de6c4 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 0fd3980 |
| Default Java | 1.8.0_91 |
| findbugs | v3.0.0 |
| Test Results | https://builds.apache.org/job/PreCommit-YARN-Build/12277/testReport/ |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell |
| Console output | https://builds.apache.org/job/PreCommit-YARN-Build/12277/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
| +1 overall |
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| 0 | reexec | 0m 19s | Docker mode activated. |
| +1 | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. |
| +1 | mvninstall | 9m 4s | trunk passed |
| +1 | compile | 0m 17s | trunk passed |
| +1 | checkstyle | 0m 14s | trunk passed |
| +1 | mvnsite | 0m 23s | trunk passed |
| +1 | mvneclipse | 0m 17s | trunk passed |
| +1 | findbugs | 0m 27s | trunk passed |
| +1 | javadoc | 0m 13s | trunk passed |
| +1 | mvninstall | 0m 16s | the patch passed |
| +1 | compile | 0m 14s | the patch passed |
| +1 | javac | 0m 14s | the patch passed |
| +1 | checkstyle | 0m 11s | hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell: The patch generated 0 new + 78 unchanged - 1 fixed = 78 total (was 79) |
| +1 | mvnsite | 0m 19s | the patch passed |
| +1 | mvneclipse | 0m 13s | the patch passed |
| +1 | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 | findbugs | 0m 31s | the patch passed |
| +1 | javadoc | 0m 10s | the patch passed |
| +1 | unit | 9m 26s | hadoop-yarn-applications-distributedshell in the patch passed. |
| +1 | asflicense | 0m 17s | The patch does not generate ASF License warnings. |
| 23m 29s |
| Subsystem | Report/Notes |
|---|---|
| Docker | Image:yetus/hadoop:9560f25 |
| JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12817263/YARN-5354.01.patch |
| JIRA Issue | |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle |
| uname | Linux af42d504b088 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 0fd3980 |
| Default Java | 1.8.0_91 |
| findbugs | v3.0.0 |
| Test Results | https://builds.apache.org/job/PreCommit-YARN-Build/12278/testReport/ |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell |
| Console output | https://builds.apache.org/job/PreCommit-YARN-Build/12278/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
I would greatly appreciate your review. Thanks!
This should work, but why not use
@Rule public final TemporaryFolder folder = new TemporaryFolder();
Or name it timelineV2StorageDir.
That seems to be the pattern used in my unit test. Has the advantage that you don't have to write cleanup code in your tearDown.
That's a good suggestion. I've updated the patch to use TemporaryFolder instead.
| +1 overall |
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| 0 | reexec | 0m 35s | Docker mode activated. |
| +1 | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. |
| +1 | mvninstall | 7m 33s | trunk passed |
| +1 | compile | 0m 17s | trunk passed |
| +1 | checkstyle | 0m 14s | trunk passed |
| +1 | mvnsite | 0m 23s | trunk passed |
| +1 | mvneclipse | 0m 17s | trunk passed |
| +1 | findbugs | 0m 26s | trunk passed |
| +1 | javadoc | 0m 14s | trunk passed |
| +1 | mvninstall | 0m 16s | the patch passed |
| +1 | compile | 0m 15s | the patch passed |
| +1 | javac | 0m 15s | the patch passed |
| +1 | checkstyle | 0m 11s | hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell: The patch generated 0 new + 78 unchanged - 1 fixed = 78 total (was 79) |
| +1 | mvnsite | 0m 21s | the patch passed |
| +1 | mvneclipse | 0m 14s | the patch passed |
| +1 | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 | findbugs | 0m 34s | the patch passed |
| +1 | javadoc | 0m 9s | the patch passed |
| +1 | unit | 9m 40s | hadoop-yarn-applications-distributedshell in the patch passed. |
| +1 | asflicense | 0m 16s | The patch does not generate ASF License warnings. |
| 22m 36s |
| Subsystem | Report/Notes |
|---|---|
| Docker | Image:yetus/hadoop:9560f25 |
| JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12817329/YARN-5354.02.patch |
| JIRA Issue | |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle |
| uname | Linux d36eaa882fe4 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / f292624 |
| Default Java | 1.8.0_91 |
| findbugs | v3.0.0 |
| Test Results | https://builds.apache.org/job/PreCommit-YARN-Build/12282/testReport/ |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell |
| Console output | https://builds.apache.org/job/PreCommit-YARN-Build/12282/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
Posted patch v.3.
Removed code that deletes the directory.
| +1 overall |
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| 0 | reexec | 0m 39s | Docker mode activated. |
| +1 | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. |
| +1 | mvninstall | 8m 46s | trunk passed |
| +1 | compile | 0m 21s | trunk passed |
| +1 | checkstyle | 0m 16s | trunk passed |
| +1 | mvnsite | 0m 26s | trunk passed |
| +1 | mvneclipse | 0m 21s | trunk passed |
| +1 | findbugs | 0m 33s | trunk passed |
| +1 | javadoc | 0m 15s | trunk passed |
| +1 | mvninstall | 0m 20s | the patch passed |
| +1 | compile | 0m 20s | the patch passed |
| +1 | javac | 0m 20s | the patch passed |
| +1 | checkstyle | 0m 14s | hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell: The patch generated 0 new + 78 unchanged - 1 fixed = 78 total (was 79) |
| +1 | mvnsite | 0m 26s | the patch passed |
| +1 | mvneclipse | 0m 16s | the patch passed |
| +1 | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 | findbugs | 0m 40s | the patch passed |
| +1 | javadoc | 0m 12s | the patch passed |
| +1 | unit | 9m 53s | hadoop-yarn-applications-distributedshell in the patch passed. |
| +1 | asflicense | 0m 18s | The patch does not generate ASF License warnings. |
| 25m 2s |
| Subsystem | Report/Notes |
|---|---|
| Docker | Image:yetus/hadoop:9560f25 |
| JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12817334/YARN-5354.03.patch |
| JIRA Issue | |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle |
| uname | Linux 34fc7c068c7c 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / f292624 |
| Default Java | 1.8.0_91 |
| findbugs | v3.0.0 |
| Test Results | https://builds.apache.org/job/PreCommit-YARN-Build/12283/testReport/ |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell |
| Console output | https://builds.apache.org/job/PreCommit-YARN-Build/12283/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
Changes LGTM. Will commit it later today.
Did not know about TemporaryFolder. Seems like a good rule to use.
Committed the latest patch to trunk.
Thanks Sangjin Lee for your contribution. And thanks Joep Rottinghuis for the review.
SUCCESS: Integrated in Hadoop-trunk-Commit #10079 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10079/)
YARN-5354. TestDistributedShell.checkTimelineV2 fails randomly for (varunsaxena: rev a3ac1c1943518c2a242dd21dac0f17f21bc3cd7f)
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/test/java/org/apache/hadoop/yarn/applications/distributedshell/TestDistributedShell.java
Posted patch v.1.
Switched to using a test-specific storage directory. I realize that the FileSystemTimelineWriterImpl default value should be handled separately. I'll file a separate JIRA.