Hadoop YARN
  1. Hadoop YARN
  2. YARN-109

.tmp file is not deleted for localized archives

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.23.3, 2.0.0-alpha
    • Fix Version/s: 3.0.0, 0.23.7, 2.1.0-beta
    • Component/s: nodemanager
    • Labels:
      None

      Description

      When archives are localized they are initially created as a .tmp file and unpacked from that file. However the .tmp file is not deleted afterwards.

      1. YARN-109-trunk.patch
        8 kB
        Mayank Bansal
      2. YARN-109-trunk-1.patch
        8 kB
        Mayank Bansal
      3. YARN-109-trunk-2.patch
        9 kB
        Mayank Bansal
      4. YARN-109-trunk-3.patch
        16 kB
        Mayank Bansal
      5. YARN-109-trunk-4.patch
        16 kB
        Mayank Bansal
      6. YARN-109-trunk-5.patch
        16 kB
        Mayank Bansal

        Activity

        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk #1384 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1384/)
        YARN-109. .tmp file is not deleted for localized archives (Mayank Bansal via bobby) (Revision 1460723)

        Result = FAILURE
        bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1460723
        Files :

        • /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/FSDownload.java
        • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestFSDownload.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1384 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1384/ ) YARN-109 . .tmp file is not deleted for localized archives (Mayank Bansal via bobby) (Revision 1460723) Result = FAILURE bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1460723 Files : /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/FSDownload.java /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestFSDownload.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk #1356 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1356/)
        YARN-109. .tmp file is not deleted for localized archives (Mayank Bansal via bobby) (Revision 1460723)

        Result = FAILURE
        bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1460723
        Files :

        • /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/FSDownload.java
        • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestFSDownload.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1356 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1356/ ) YARN-109 . .tmp file is not deleted for localized archives (Mayank Bansal via bobby) (Revision 1460723) Result = FAILURE bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1460723 Files : /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/FSDownload.java /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestFSDownload.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-0.23-Build #565 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/565/)
        svn merge -c 1460723 FIXES: YARN-109. .tmp file is not deleted for localized archives (Mayank Bansal via bobby) (Revision 1460734)

        Result = UNSTABLE
        bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1460734
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-yarn-project/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/FSDownload.java
        • /hadoop/common/branches/branch-0.23/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestFSDownload.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #565 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/565/ ) svn merge -c 1460723 FIXES: YARN-109 . .tmp file is not deleted for localized archives (Mayank Bansal via bobby) (Revision 1460734) Result = UNSTABLE bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1460734 Files : /hadoop/common/branches/branch-0.23/hadoop-yarn-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/FSDownload.java /hadoop/common/branches/branch-0.23/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestFSDownload.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Yarn-trunk #167 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/167/)
        YARN-109. .tmp file is not deleted for localized archives (Mayank Bansal via bobby) (Revision 1460723)

        Result = SUCCESS
        bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1460723
        Files :

        • /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/FSDownload.java
        • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestFSDownload.java
        Show
        Hudson added a comment - Integrated in Hadoop-Yarn-trunk #167 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/167/ ) YARN-109 . .tmp file is not deleted for localized archives (Mayank Bansal via bobby) (Revision 1460723) Result = SUCCESS bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1460723 Files : /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/FSDownload.java /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestFSDownload.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-trunk-Commit #3521 (See https://builds.apache.org/job/Hadoop-trunk-Commit/3521/)
        YARN-109. .tmp file is not deleted for localized archives (Mayank Bansal via bobby) (Revision 1460723)

        Result = SUCCESS
        bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1460723
        Files :

        • /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/FSDownload.java
        • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestFSDownload.java
        Show
        Hudson added a comment - Integrated in Hadoop-trunk-Commit #3521 (See https://builds.apache.org/job/Hadoop-trunk-Commit/3521/ ) YARN-109 . .tmp file is not deleted for localized archives (Mayank Bansal via bobby) (Revision 1460723) Result = SUCCESS bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1460723 Files : /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/FSDownload.java /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestFSDownload.java
        Hide
        Robert Joseph Evans added a comment -

        Thanks Mayank,

        I put this into trunk, branch-2, and branch-0.23. Keep up the good work.

        Show
        Robert Joseph Evans added a comment - Thanks Mayank, I put this into trunk, branch-2, and branch-0.23. Keep up the good work.
        Hide
        Robert Joseph Evans added a comment -

        The changes look good to me. The test seems a bit brittle, looking to be sure that a .tmp is not there, but I don't know any other way to do it. +1

        I'll check it in.

        Show
        Robert Joseph Evans added a comment - The changes look good to me. The test seems a bit brittle, looking to be sure that a .tmp is not there, but I don't know any other way to do it. +1 I'll check it in.
        Hide
        Mayank Bansal added a comment -

        Robert Joseph Evans
        Thanks for the review.
        Do you mind committing it as well?

        Thanks,
        Mayank

        Show
        Mayank Bansal added a comment - Robert Joseph Evans Thanks for the review. Do you mind committing it as well? Thanks, Mayank
        Hide
        Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12574891/YARN-109-trunk-5.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 1 new or modified test files.

        +1 tests included appear to have a timeout.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-YARN-Build/563//testReport/
        Console output: https://builds.apache.org/job/PreCommit-YARN-Build/563//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12574891/YARN-109-trunk-5.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 tests included appear to have a timeout. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-YARN-Build/563//testReport/ Console output: https://builds.apache.org/job/PreCommit-YARN-Build/563//console This message is automatically generated.
        Hide
        Mayank Bansal added a comment -

        Robert Joseph Evans
        Sure I will file a new JIRA for that.

        Thanks,
        Mayank

        Show
        Mayank Bansal added a comment - Robert Joseph Evans Sure I will file a new JIRA for that. Thanks, Mayank
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12574885/YARN-109-trunk-4.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 1 new or modified test files.

        +1 tests included appear to have a timeout.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common:

        org.apache.hadoop.yarn.util.TestFSDownload

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-YARN-Build/559//testReport/
        Console output: https://builds.apache.org/job/PreCommit-YARN-Build/559//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12574885/YARN-109-trunk-4.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 tests included appear to have a timeout. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common: org.apache.hadoop.yarn.util.TestFSDownload +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-YARN-Build/559//testReport/ Console output: https://builds.apache.org/job/PreCommit-YARN-Build/559//console This message is automatically generated.
        Hide
        Robert Joseph Evans added a comment -

        That is fine with me. My concern was mostly with Windows support. tar, zip, jar, etc. should be there, but bash may not be. So if you want to file a new JIRA that is fine, if not you can just wait for windows support to be merged in and see if it breaks.

        Show
        Robert Joseph Evans added a comment - That is fine with me. My concern was mostly with Windows support. tar, zip, jar, etc. should be there, but bash may not be. So if you want to file a new JIRA that is fine, if not you can just wait for windows support to be merged in and see if it breaks.
        Hide
        Mayank Bansal added a comment -

        Robert Joseph Evans

        Thanks for the comments and review. Updating the new patch.

        For adding the jar/tar/zip files in to repo I am not sure at this moment as i see in other cases it is also generating it on the fly.
        However I think your point is valid.
        I will recommend to do that in separate JIRA for everything to maintain consistency.

        Thoughts?

        Thanks,
        Mayank

        Show
        Mayank Bansal added a comment - Robert Joseph Evans Thanks for the comments and review. Updating the new patch. For adding the jar/tar/zip files in to repo I am not sure at this moment as i see in other cases it is also generating it on the fly. However I think your point is valid. I will recommend to do that in separate JIRA for everything to maintain consistency. Thoughts? Thanks, Mayank
        Hide
        Omkar Vinit Joshi added a comment -

        Mayank Bansal Thanks for updating it. Now it looks good.

        Show
        Omkar Vinit Joshi added a comment - Mayank Bansal Thanks for updating it. Now it looks good.
        Hide
        Robert Joseph Evans added a comment -

        The findbugs is complaining that you are ignoring the return value of the delete call. It should not be a problem so either use the return value to log a warning when it fails or update the findbugs filter to filter out the error.

        The -1 for the test timeouts is caused by a bug in the script used to detect these, so you can either ignore it, or add a timeout to any @Test that appears in the patch file, including the ones you didn't add .

        In the test, please uncomment the lines to cleanup after the test. Are they causing a problem for the test to pass? or was it just for debugging?

        Also I personally would prefer to have a few small jar/tar/zip files checked into the repository instead of generating them on they fly for the test. It will speed up the test and have less dependencies on the system being set up with the exact commands, i.e. bash for windows support. Although if you don't feel like changing it I am fine with that too, most of those commands are used by FSDownload already so it is not that critical.

        Show
        Robert Joseph Evans added a comment - The findbugs is complaining that you are ignoring the return value of the delete call. It should not be a problem so either use the return value to log a warning when it fails or update the findbugs filter to filter out the error. The -1 for the test timeouts is caused by a bug in the script used to detect these, so you can either ignore it, or add a timeout to any @Test that appears in the patch file, including the ones you didn't add . In the test, please uncomment the lines to cleanup after the test. Are they causing a problem for the test to pass? or was it just for debugging? Also I personally would prefer to have a few small jar/tar/zip files checked into the repository instead of generating them on they fly for the test. It will speed up the test and have less dependencies on the system being set up with the exact commands, i.e. bash for windows support. Although if you don't feel like changing it I am fine with that too, most of those commands are used by FSDownload already so it is not that critical.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12574743/YARN-109-trunk-3.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 1 new or modified test files.

        -1 one of tests included doesn't have a timeout.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        -1 findbugs. The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-YARN-Build/555//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-YARN-Build/555//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-common.html
        Console output: https://builds.apache.org/job/PreCommit-YARN-Build/555//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12574743/YARN-109-trunk-3.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. -1 one of tests included doesn't have a timeout. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. -1 findbugs . The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-YARN-Build/555//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-YARN-Build/555//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-common.html Console output: https://builds.apache.org/job/PreCommit-YARN-Build/555//console This message is automatically generated.
        Hide
        Mayank Bansal added a comment -

        Omkar Vinit Joshi
        There are no changes in call its just I formatted that with hadoop formatter.

        I modified the unpack for all the cases.

        I added couple of more test methods to test some more cases , Looks to me enough for this change. Not necessary to test all combinations.

        I add the timeouts.

        Thanks,
        Mayank

        Show
        Mayank Bansal added a comment - Omkar Vinit Joshi There are no changes in call its just I formatted that with hadoop formatter. I modified the unpack for all the cases. I added couple of more test methods to test some more cases , Looks to me enough for this change. Not necessary to test all combinations. I add the timeouts. Thanks, Mayank
        Hide
        Omkar Vinit Joshi added a comment -

        Mayank Bansal I looked at the patch. Even though it is not modifying call method logically but still I see changes there.

        Can we simply keep a flag to track all the places in unpack method where renameTo was called (To indicate not to delete file as it is already deleted). and towards the end before returning we can delete file based on flag value. For file deletion we can even use File.delete(). I guess this will make it cleaner and will keep the tmp file handling logic intact as we need to log an error message in case we are unable to delete the .tmp file.

        The code is still not handling situation where resource.getType() == ARCHIVE and it is .jar or if it is PATTERN and .zip file.

        Also I see that entire test code is only testing if the file is a tar file and ARCHIVE type but not for other extensions / TYPE. I think we need to test all the combinations ( .zip , .tar, .jar ) and it is specified as FILE, PATTERN or ARCHIVE.

        Also for the test I think you will have to add a timeout parameter. @Test(timeout=<value>) to avoid -1.

        Please let me know your thoughts on this.

        Show
        Omkar Vinit Joshi added a comment - Mayank Bansal I looked at the patch. Even though it is not modifying call method logically but still I see changes there. Can we simply keep a flag to track all the places in unpack method where renameTo was called (To indicate not to delete file as it is already deleted). and towards the end before returning we can delete file based on flag value. For file deletion we can even use File.delete(). I guess this will make it cleaner and will keep the tmp file handling logic intact as we need to log an error message in case we are unable to delete the .tmp file. The code is still not handling situation where resource.getType() == ARCHIVE and it is .jar or if it is PATTERN and .zip file. Also I see that entire test code is only testing if the file is a tar file and ARCHIVE type but not for other extensions / TYPE. I think we need to test all the combinations ( .zip , .tar, .jar ) and it is specified as FILE, PATTERN or ARCHIVE. Also for the test I think you will have to add a timeout parameter. @Test(timeout=<value>) to avoid -1. Please let me know your thoughts on this.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12574666/YARN-109-trunk-2.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 1 new or modified test files.

        -1 one of tests included doesn't have a timeout.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-YARN-Build/551//testReport/
        Console output: https://builds.apache.org/job/PreCommit-YARN-Build/551//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12574666/YARN-109-trunk-2.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. -1 one of tests included doesn't have a timeout. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-YARN-Build/551//testReport/ Console output: https://builds.apache.org/job/PreCommit-YARN-Build/551//console This message is automatically generated.
        Hide
        Mayank Bansal added a comment -

        Omkar Vinit Joshi
        Good points.
        Adding another patch.

        Thanks,
        Mayank

        Show
        Mayank Bansal added a comment - Omkar Vinit Joshi Good points. Adding another patch. Thanks, Mayank
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12574444/YARN-109-trunk-1.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 1 new or modified test files.

        -1 one of tests included doesn't have a timeout.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-YARN-Build/545//testReport/
        Console output: https://builds.apache.org/job/PreCommit-YARN-Build/545//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12574444/YARN-109-trunk-1.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. -1 one of tests included doesn't have a timeout. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-YARN-Build/545//testReport/ Console output: https://builds.apache.org/job/PreCommit-YARN-Build/545//console This message is automatically generated.
        Hide
        Omkar Vinit Joshi added a comment -

        Mayank Bansal can we move the file deletion logic to the unpack method? I think that would be the right place to delete the file as it is already happening for resource.getType() == FILE. (during renameTo call). The current patch won't handle the situation where
        1) user has specified the file type as ARCHIVE but the extension is neither of these (.jar,.tar,.tar.gz,.tgz) then it will add an incorrect error log message about file not deleted.
        2) If the user has specified the file type as PATTERN then this will not remove the file.

        Let me know what are your thoughts.

        Show
        Omkar Vinit Joshi added a comment - Mayank Bansal can we move the file deletion logic to the unpack method? I think that would be the right place to delete the file as it is already happening for resource.getType() == FILE. (during renameTo call). The current patch won't handle the situation where 1) user has specified the file type as ARCHIVE but the extension is neither of these (.jar,.tar,.tar.gz,.tgz) then it will add an incorrect error log message about file not deleted. 2) If the user has specified the file type as PATTERN then this will not remove the file. Let me know what are your thoughts.
        Hide
        Mayank Bansal added a comment -

        Fixing Warning

        Show
        Mayank Bansal added a comment - Fixing Warning
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12574430/YARN-109-trunk.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 1 new or modified test files.

        -1 one of tests included doesn't have a timeout.

        -1 javac. The applied patch generated 1362 javac compiler warnings (more than the trunk's current 1361 warnings).

        +1 javadoc. The javadoc tool did not generate any warning messages.

        -1 eclipse:eclipse. The patch failed to build with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-YARN-Build/544//testReport/
        Javac warnings: https://builds.apache.org/job/PreCommit-YARN-Build/544//artifact/trunk/patchprocess/diffJavacWarnings.txt
        Console output: https://builds.apache.org/job/PreCommit-YARN-Build/544//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12574430/YARN-109-trunk.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. -1 one of tests included doesn't have a timeout. -1 javac . The applied patch generated 1362 javac compiler warnings (more than the trunk's current 1361 warnings). +1 javadoc . The javadoc tool did not generate any warning messages. -1 eclipse:eclipse . The patch failed to build with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-YARN-Build/544//testReport/ Javac warnings: https://builds.apache.org/job/PreCommit-YARN-Build/544//artifact/trunk/patchprocess/diffJavacWarnings.txt Console output: https://builds.apache.org/job/PreCommit-YARN-Build/544//console This message is automatically generated.
        Hide
        Mayank Bansal added a comment -

        Attaching Patch

        Thanks,
        Mayank

        Show
        Mayank Bansal added a comment - Attaching Patch Thanks, Mayank
        Hide
        Omkar Vinit Joshi added a comment -

        Mayank Bansal Are you still looking into this issue? or else I would like to take over.

        Show
        Omkar Vinit Joshi added a comment - Mayank Bansal Are you still looking into this issue? or else I would like to take over.

          People

          • Assignee:
            Mayank Bansal
            Reporter:
            Jason Lowe
          • Votes:
            0 Vote for this issue
            Watchers:
            10 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development