Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-8844

TestHDFSCLI does not cleanup the test directory

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: test
    • Labels:
      None
    • Target Version/s:

      Description

      If TestHDFSCLI is executed twice without mvn clean, the second try fails. Here are the failing test cases:

      2015-07-31 21:35:17,654 [main] INFO  cli.CLITestHelper (CLITestHelper.java:displayResults(231)) - Failing tests:
      2015-07-31 21:35:17,654 [main] INFO  cli.CLITestHelper (CLITestHelper.java:displayResults(232)) - --------------
      2015-07-31 21:35:17,654 [main] INFO  cli.CLITestHelper (CLITestHelper.java:displayResults(238)) - 226: get: getting non existent(absolute path)
      2015-07-31 21:35:17,654 [main] INFO  cli.CLITestHelper (CLITestHelper.java:displayResults(238)) - 227: get: getting non existent file(relative path)
      2015-07-31 21:35:17,654 [main] INFO  cli.CLITestHelper (CLITestHelper.java:displayResults(238)) - 228: get: Test for hdfs:// path - getting non existent
      2015-07-31 21:35:17,654 [main] INFO  cli.CLITestHelper (CLITestHelper.java:displayResults(238)) - 229: get: Test for Namenode's path - getting non existent
      2015-07-31 21:35:17,654 [main] INFO  cli.CLITestHelper (CLITestHelper.java:displayResults(238)) - 250: copyToLocal: non existent relative path
      2015-07-31 21:35:17,654 [main] INFO  cli.CLITestHelper (CLITestHelper.java:displayResults(238)) - 251: copyToLocal: non existent absolute path
      2015-07-31 21:35:17,655 [main] INFO  cli.CLITestHelper (CLITestHelper.java:displayResults(238)) - 252: copyToLocal: Test for hdfs:// path - non existent file/directory
      2015-07-31 21:35:17,655 [main] INFO  cli.CLITestHelper (CLITestHelper.java:displayResults(238)) - 253: copyToLocal: Test for Namenode's path - non existent file/directory
      
      1. HDFS-8844.001.patch
        0.7 kB
        Masatake Iwasaki

        Activity

        Hide
        iwasakims Masatake Iwasaki added a comment -

        Failed tests are copying non-existent file to CLITEST_DATA/file but expected output could not be given if the tartget CLITEST_DATA/file exists. The cause is test for getmerge creating CLITEST_DATA/file but not deleting it.

            <test>
              <description>getmerge</description>
              <test-commands>
                <command>-fs NAMENODE -mkdir -p /user/USERNAME/dir1</command>
                <command>-fs NAMENODE -copyFromLocal CLITEST_DATA/data15bytes /user/USERNAME/dir1</command>
                <command>-fs NAMENODE -copyFromLocal CLITEST_DATA/data30bytes /user/USERNAME/dir1</command>
                <command>-fs NAMENODE -getmerge /user/USERNAME/dir1 CLITEST_DATA/file</command>
                <command>-cat CLITEST_DATA/file</command>
              </test-commands>
              <cleanup-commands>
                <command>-fs NAMENODE -rm -r /user/USERNAME CLITEST_DATA/file</command>
                <command>rm data</command>
        
        Show
        iwasakims Masatake Iwasaki added a comment - Failed tests are copying non-existent file to CLITEST_DATA/file but expected output could not be given if the tartget CLITEST_DATA/file exists. The cause is test for getmerge creating CLITEST_DATA/file but not deleting it. <test> <description>getmerge</description> <test-commands> <command>-fs NAMENODE -mkdir -p /user/USERNAME/dir1</command> <command>-fs NAMENODE -copyFromLocal CLITEST_DATA/data15bytes /user/USERNAME/dir1</command> <command>-fs NAMENODE -copyFromLocal CLITEST_DATA/data30bytes /user/USERNAME/dir1</command> <command>-fs NAMENODE -getmerge /user/USERNAME/dir1 CLITEST_DATA/file</command> <command>-cat CLITEST_DATA/file</command> </test-commands> <cleanup-commands> <command>-fs NAMENODE -rm -r /user/USERNAME CLITEST_DATA/file</command> <command>rm data</command>
        Hide
        iwasakims Masatake Iwasaki added a comment -
        Expected output:   [get: `.*': No such file or directory]
          Actual output:   [get: `/iwasakims/srcs/hadoop/hadoop-hdfs-project/hadoop-hdfs/target/test-classes//file': File exists
        
        Show
        iwasakims Masatake Iwasaki added a comment - Expected output: [get: `.*': No such file or directory] Actual output: [get: `/iwasakims/srcs/hadoop/hadoop-hdfs-project/hadoop-hdfs/target/test-classes//file': File exists
        Hide
        hadoopqa Hadoop QA added a comment -



        +1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 5m 27s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
        +1 javac 7m 37s There were no new javac warning messages.
        +1 release audit 0m 21s The applied patch does not increase the total number of release audit warnings.
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 install 1m 23s mvn install still works.
        +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse.
        +1 native 0m 57s Pre-build of native portion
        +1 hdfs tests 162m 56s Tests passed in hadoop-hdfs.
            179m 17s  



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12748344/HDFS-8844.001.patch
        Optional Tests javac unit
        git revision trunk / 90b5104
        hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/11885/artifact/patchprocess/testrun_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11885/testReport/
        Java 1.7.0_55
        uname Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11885/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 pre-patch 5m 27s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 7m 37s There were no new javac warning messages. +1 release audit 0m 21s The applied patch does not increase the total number of release audit warnings. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 23s mvn install still works. +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse. +1 native 0m 57s Pre-build of native portion +1 hdfs tests 162m 56s Tests passed in hadoop-hdfs.     179m 17s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12748344/HDFS-8844.001.patch Optional Tests javac unit git revision trunk / 90b5104 hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/11885/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11885/testReport/ Java 1.7.0_55 uname Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11885/console This message was automatically generated.
        Hide
        cmccabe Colin P. McCabe added a comment -

        +1. Thanks, Masatake Iwasaki.

        Show
        cmccabe Colin P. McCabe added a comment - +1. Thanks, Masatake Iwasaki .
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-trunk-Commit #8260 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8260/)
        HDFS-8844. TestHDFSCLI does not cleanup the test directory (Masatake Iwasaki via Colin P. McCabe) (cmccabe: rev c95993cbaf51e2925ea9b1b95cf4f0d879e66489)

        • hadoop-hdfs-project/hadoop-hdfs/src/test/resources/testHDFSConf.xml
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8260 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8260/ ) HDFS-8844 . TestHDFSCLI does not cleanup the test directory (Masatake Iwasaki via Colin P. McCabe) (cmccabe: rev c95993cbaf51e2925ea9b1b95cf4f0d879e66489) hadoop-hdfs-project/hadoop-hdfs/src/test/resources/testHDFSConf.xml hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        ABORTED: Integrated in Hadoop-Yarn-trunk #1008 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1008/)
        HDFS-8844. TestHDFSCLI does not cleanup the test directory (Masatake Iwasaki via Colin P. McCabe) (cmccabe: rev c95993cbaf51e2925ea9b1b95cf4f0d879e66489)

        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/test/resources/testHDFSConf.xml
        Show
        hudson Hudson added a comment - ABORTED: Integrated in Hadoop-Yarn-trunk #1008 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1008/ ) HDFS-8844 . TestHDFSCLI does not cleanup the test directory (Masatake Iwasaki via Colin P. McCabe) (cmccabe: rev c95993cbaf51e2925ea9b1b95cf4f0d879e66489) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/resources/testHDFSConf.xml
        Hide
        hudson Hudson added a comment -

        ABORTED: Integrated in Hadoop-Yarn-trunk-Java8 #278 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/278/)
        HDFS-8844. TestHDFSCLI does not cleanup the test directory (Masatake Iwasaki via Colin P. McCabe) (cmccabe: rev c95993cbaf51e2925ea9b1b95cf4f0d879e66489)

        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/test/resources/testHDFSConf.xml
        Show
        hudson Hudson added a comment - ABORTED: Integrated in Hadoop-Yarn-trunk-Java8 #278 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/278/ ) HDFS-8844 . TestHDFSCLI does not cleanup the test directory (Masatake Iwasaki via Colin P. McCabe) (cmccabe: rev c95993cbaf51e2925ea9b1b95cf4f0d879e66489) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/resources/testHDFSConf.xml
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #267 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/267/)
        HDFS-8844. TestHDFSCLI does not cleanup the test directory (Masatake Iwasaki via Colin P. McCabe) (cmccabe: rev c95993cbaf51e2925ea9b1b95cf4f0d879e66489)

        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/test/resources/testHDFSConf.xml
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #267 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/267/ ) HDFS-8844 . TestHDFSCLI does not cleanup the test directory (Masatake Iwasaki via Colin P. McCabe) (cmccabe: rev c95993cbaf51e2925ea9b1b95cf4f0d879e66489) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/resources/testHDFSConf.xml
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk #2205 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2205/)
        HDFS-8844. TestHDFSCLI does not cleanup the test directory (Masatake Iwasaki via Colin P. McCabe) (cmccabe: rev c95993cbaf51e2925ea9b1b95cf4f0d879e66489)

        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/test/resources/testHDFSConf.xml
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2205 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2205/ ) HDFS-8844 . TestHDFSCLI does not cleanup the test directory (Masatake Iwasaki via Colin P. McCabe) (cmccabe: rev c95993cbaf51e2925ea9b1b95cf4f0d879e66489) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/resources/testHDFSConf.xml
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-Mapreduce-trunk-Java8 #275 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/275/)
        HDFS-8844. TestHDFSCLI does not cleanup the test directory (Masatake Iwasaki via Colin P. McCabe) (cmccabe: rev c95993cbaf51e2925ea9b1b95cf4f0d879e66489)

        • hadoop-hdfs-project/hadoop-hdfs/src/test/resources/testHDFSConf.xml
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Mapreduce-trunk-Java8 #275 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/275/ ) HDFS-8844 . TestHDFSCLI does not cleanup the test directory (Masatake Iwasaki via Colin P. McCabe) (cmccabe: rev c95993cbaf51e2925ea9b1b95cf4f0d879e66489) hadoop-hdfs-project/hadoop-hdfs/src/test/resources/testHDFSConf.xml hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2224 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2224/)
        HDFS-8844. TestHDFSCLI does not cleanup the test directory (Masatake Iwasaki via Colin P. McCabe) (cmccabe: rev c95993cbaf51e2925ea9b1b95cf4f0d879e66489)

        • hadoop-hdfs-project/hadoop-hdfs/src/test/resources/testHDFSConf.xml
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2224 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2224/ ) HDFS-8844 . TestHDFSCLI does not cleanup the test directory (Masatake Iwasaki via Colin P. McCabe) (cmccabe: rev c95993cbaf51e2925ea9b1b95cf4f0d879e66489) hadoop-hdfs-project/hadoop-hdfs/src/test/resources/testHDFSConf.xml hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt

          People

          • Assignee:
            iwasakims Masatake Iwasaki
            Reporter:
            ajisakaa Akira Ajisaka
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development