Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-14247

FileContextMainOperationsBaseTest should clean up test root path

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 2.8.0
    • Fix Version/s: 2.9.0, 3.0.0-alpha4, 2.8.2
    • Component/s: fs, test
    • Labels:
      None

      Description

      fileContextTestHelper will create random test root directory for each test case, say $RANDOM. After test, we only delete the $RANDOM/test directory in tearDown(). This will leave dangling directories under the test-dir directory. For S3A with S3Guard enabled, those useless test directories will stay in DynamoDB forever. Let's clear out the test root path, which is (generated randomly and) safe to remove.

        Activity

        Hide
        liuml07 Mingliang Liu added a comment -

        Steve Loughran can you have a quick look?

        Show
        liuml07 Mingliang Liu added a comment - Steve Loughran can you have a quick look?
        Hide
        hadoopqa Hadoop QA added a comment -
        +1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 18s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 14m 38s trunk passed
        +1 compile 16m 45s trunk passed
        +1 checkstyle 0m 39s trunk passed
        +1 mvnsite 1m 6s trunk passed
        +1 mvneclipse 0m 19s trunk passed
        +1 findbugs 1m 27s trunk passed
        +1 javadoc 0m 48s trunk passed
        +1 mvninstall 0m 37s the patch passed
        +1 compile 14m 20s the patch passed
        +1 javac 14m 20s the patch passed
        +1 checkstyle 0m 36s hadoop-common-project/hadoop-common: The patch generated 0 new + 74 unchanged - 1 fixed = 74 total (was 75)
        +1 mvnsite 1m 2s the patch passed
        +1 mvneclipse 0m 19s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 1m 33s the patch passed
        +1 javadoc 0m 50s the patch passed
        +1 unit 8m 4s hadoop-common in the patch passed.
        +1 asflicense 0m 34s The patch does not generate ASF License warnings.
        65m 53s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:a9ad5d6
        JIRA Issue HADOOP-14247
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12860762/HADOOP-14247.000.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux ad07d963a7a4 3.13.0-103-generic #150-Ubuntu SMP Thu Nov 24 10:34:17 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / cd014d5
        Default Java 1.8.0_121
        findbugs v3.0.0
        Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/11948/testReport/
        modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11948/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 18s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 14m 38s trunk passed +1 compile 16m 45s trunk passed +1 checkstyle 0m 39s trunk passed +1 mvnsite 1m 6s trunk passed +1 mvneclipse 0m 19s trunk passed +1 findbugs 1m 27s trunk passed +1 javadoc 0m 48s trunk passed +1 mvninstall 0m 37s the patch passed +1 compile 14m 20s the patch passed +1 javac 14m 20s the patch passed +1 checkstyle 0m 36s hadoop-common-project/hadoop-common: The patch generated 0 new + 74 unchanged - 1 fixed = 74 total (was 75) +1 mvnsite 1m 2s the patch passed +1 mvneclipse 0m 19s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 33s the patch passed +1 javadoc 0m 50s the patch passed +1 unit 8m 4s hadoop-common in the patch passed. +1 asflicense 0m 34s The patch does not generate ASF License warnings. 65m 53s Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HADOOP-14247 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12860762/HADOOP-14247.000.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux ad07d963a7a4 3.13.0-103-generic #150-Ubuntu SMP Thu Nov 24 10:34:17 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / cd014d5 Default Java 1.8.0_121 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/11948/testReport/ modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11948/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        liuml07 Mingliang Liu added a comment -

        Tested S3A in us-west-1 region. Tested HDFS/WebHDFS/SWebHDFS/LocalFS.

        Show
        liuml07 Mingliang Liu added a comment - Tested S3A in us-west-1 region. Tested HDFS/WebHDFS/SWebHDFS/LocalFS.
        Hide
        stevel@apache.org Steve Loughran added a comment -

        I don't think teardown should have an assert, though it could warn. If it raises exceptions, then teardown failures will end up hiding any failure in the test code itself

        Show
        stevel@apache.org Steve Loughran added a comment - I don't think teardown should have an assert, though it could warn. If it raises exceptions, then teardown failures will end up hiding any failure in the test code itself
        Hide
        liuml07 Mingliang Liu added a comment -

        So Steve Loughran are you +1 on the patch?

        Show
        liuml07 Mingliang Liu added a comment - So Steve Loughran are you +1 on the patch?
        Hide
        stevel@apache.org Steve Loughran added a comment -

        I'd like the assert check pulled from teardown. Otherwise the sequence of: test failure, teardown failure, will only show the teardown exception, not the underlying test failure. I'd end up having to edit the code, comment out the assert and rerun. Catch and log, fine: but teardowns mustn't raise exceptions

        Show
        stevel@apache.org Steve Loughran added a comment - I'd like the assert check pulled from teardown. Otherwise the sequence of: test failure, teardown failure, will only show the teardown exception, not the underlying test failure. I'd end up having to edit the code, comment out the assert and rerun. Catch and log, fine: but teardowns mustn't raise exceptions
        Hide
        stevel@apache.org Steve Loughran added a comment -

        sorry, I was mixed up.

        +1

        Show
        stevel@apache.org Steve Loughran added a comment - sorry, I was mixed up. +1
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11490 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11490/)
        HADOOP-14247. FileContextMainOperationsBaseTest should clean up test (liuml07: rev 13c766b62c48ae07849f7f1762ef6883239aef54)

        • (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/FileContextMainOperationsBaseTest.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11490 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11490/ ) HADOOP-14247 . FileContextMainOperationsBaseTest should clean up test (liuml07: rev 13c766b62c48ae07849f7f1762ef6883239aef54) (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/FileContextMainOperationsBaseTest.java
        Hide
        liuml07 Mingliang Liu added a comment -

        Committed to branch-2.8 through trunk branches. Thanks for your review and discussion Steve Loughran.

        Show
        liuml07 Mingliang Liu added a comment - Committed to branch-2.8 through trunk branches. Thanks for your review and discussion Steve Loughran .
        Hide
        vinodkv Vinod Kumar Vavilapalli added a comment -

        2.8.1 became a security release. Moving fix-version to 2.8.2 after the fact.

        Show
        vinodkv Vinod Kumar Vavilapalli added a comment - 2.8.1 became a security release. Moving fix-version to 2.8.2 after the fact.

          People

          • Assignee:
            liuml07 Mingliang Liu
            Reporter:
            liuml07 Mingliang Liu
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development