Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-13272

ViewFileSystem should support storage policy related API

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.0.0-alpha1
    • Component/s: fs, viewfs
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Current ViewFileSystem does not support storage policy related API, it will throw UnsupportedOperationException.

      1. HADOOP-13272.001.patch
        14 kB
        Peter Shi
      2. HADOOP-13272.002.patch
        14 kB
        Chris Douglas

        Activity

        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-trunk-Commit #10140 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10140/)
        HADOOP-13272. ViewFileSystem should support storage policy related API. (cdouglas: rev 22a9a6b0d412f31ea0f15ac389e56b9de9eeadb1)

        • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ChRootedFileSystem.java
        • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/viewfs/ViewFileSystemBaseTest.java
        • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ViewFileSystem.java
        • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/viewfs/TestChRootedFileSystem.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-trunk-Commit #10140 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10140/ ) HADOOP-13272 . ViewFileSystem should support storage policy related API. (cdouglas: rev 22a9a6b0d412f31ea0f15ac389e56b9de9eeadb1) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ChRootedFileSystem.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/viewfs/ViewFileSystemBaseTest.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ViewFileSystem.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/viewfs/TestChRootedFileSystem.java
        Hide
        chris.douglas Chris Douglas added a comment -

        I committed this. Thanks, Peter!

        Show
        chris.douglas Chris Douglas added a comment - I committed this. Thanks, Peter!
        Hide
        chris.douglas Chris Douglas added a comment -

        TestHttpServerLifecycle succeeded. The error is not caused by the patch:

        Results :
        
        Tests run: 3432, Failures: 0, Errors: 0, Skipped: 84
        
        [INFO] ------------------------------------------------------------------------
        [INFO] BUILD FAILURE
        [INFO] ------------------------------------------------------------------------
        [INFO] Total time: 21:33.873s
        [INFO] Finished at: Sat Jul 23 00:45:19 UTC 2016
        [INFO] Final Memory: 24M/307M
        [INFO] ------------------------------------------------------------------------
        [ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.17:test (default-test) on project hadoop-common: There was a timeout or other error in the fork -> [Help 1]
        
        Show
        chris.douglas Chris Douglas added a comment - TestHttpServerLifecycle succeeded. The error is not caused by the patch: Results : Tests run: 3432, Failures: 0, Errors: 0, Skipped: 84 [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 21:33.873s [INFO] Finished at: Sat Jul 23 00:45:19 UTC 2016 [INFO] Final Memory: 24M/307M [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.17:test (default-test) on project hadoop-common: There was a timeout or other error in the fork -> [Help 1]
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 28s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
        +1 mvninstall 7m 21s trunk passed
        +1 compile 8m 4s trunk passed
        +1 checkstyle 1m 27s trunk passed
        +1 mvnsite 2m 5s trunk passed
        +1 mvneclipse 0m 13s trunk passed
        +1 findbugs 1m 23s trunk passed
        +1 javadoc 0m 47s trunk passed
        +1 mvninstall 0m 41s the patch passed
        +1 compile 6m 56s the patch passed
        +1 javac 6m 56s the patch passed
        +1 checkstyle 0m 25s hadoop-common-project/hadoop-common: The patch generated 0 new + 174 unchanged - 1 fixed = 174 total (was 175)
        +1 mvnsite 0m 57s the patch passed
        +1 mvneclipse 0m 12s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 1m 28s the patch passed
        +1 javadoc 0m 46s the patch passed
        -1 unit 21m 42s hadoop-common in the patch failed.
        +1 asflicense 0m 22s The patch does not generate ASF License warnings.
        57m 6s



        Reason Tests
        Timed out junit tests org.apache.hadoop.http.TestHttpServerLifecycle



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12819747/HADOOP-13272.002.patch
        JIRA Issue HADOOP-13272
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 6cfba23dbc59 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / c6e3a00
        Default Java 1.8.0_91
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HADOOP-Build/10067/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
        Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10067/testReport/
        modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10067/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 28s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. +1 mvninstall 7m 21s trunk passed +1 compile 8m 4s trunk passed +1 checkstyle 1m 27s trunk passed +1 mvnsite 2m 5s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 1m 23s trunk passed +1 javadoc 0m 47s trunk passed +1 mvninstall 0m 41s the patch passed +1 compile 6m 56s the patch passed +1 javac 6m 56s the patch passed +1 checkstyle 0m 25s hadoop-common-project/hadoop-common: The patch generated 0 new + 174 unchanged - 1 fixed = 174 total (was 175) +1 mvnsite 0m 57s the patch passed +1 mvneclipse 0m 12s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 28s the patch passed +1 javadoc 0m 46s the patch passed -1 unit 21m 42s hadoop-common in the patch failed. +1 asflicense 0m 22s The patch does not generate ASF License warnings. 57m 6s Reason Tests Timed out junit tests org.apache.hadoop.http.TestHttpServerLifecycle Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12819747/HADOOP-13272.002.patch JIRA Issue HADOOP-13272 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 6cfba23dbc59 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / c6e3a00 Default Java 1.8.0_91 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HADOOP-Build/10067/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10067/testReport/ modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10067/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        chris.douglas Chris Douglas added a comment -

        +1 lgtm

        Posting some minor modifications to avoid collapsed imports, move testInternalGetAllStoragePolicies to ViewFIleSystemBaseTest, and resolve checkstyle errors. Will commit if Jenkins comes back clean.

        Show
        chris.douglas Chris Douglas added a comment - +1 lgtm Posting some minor modifications to avoid collapsed imports, move testInternalGetAllStoragePolicies to ViewFIleSystemBaseTest , and resolve checkstyle errors. Will commit if Jenkins comes back clean.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 26s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
        0 mvndep 0m 13s Maven dependency ordering for branch
        +1 mvninstall 8m 32s trunk passed
        +1 compile 7m 22s trunk passed
        +1 checkstyle 1m 24s trunk passed
        +1 mvnsite 1m 57s trunk passed
        +1 mvneclipse 0m 26s trunk passed
        +1 findbugs 3m 15s trunk passed
        +1 javadoc 1m 40s trunk passed
        0 mvndep 0m 13s Maven dependency ordering for patch
        +1 mvninstall 1m 30s the patch passed
        +1 compile 7m 6s the patch passed
        +1 javac 7m 6s the patch passed
        -1 checkstyle 1m 24s root: The patch generated 7 new + 181 unchanged - 1 fixed = 188 total (was 182)
        +1 mvnsite 1m 49s the patch passed
        +1 mvneclipse 0m 29s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 3m 29s the patch passed
        +1 javadoc 1m 51s the patch passed
        -1 unit 7m 52s hadoop-common in the patch failed.
        -1 unit 65m 10s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 23s The patch does not generate ASF License warnings.
        117m 25s



        Reason Tests
        Failed junit tests hadoop.metrics2.impl.TestGangliaMetrics
          hadoop.hdfs.server.blockmanagement.TestReplicationPolicy



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:e2f6409
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12810976/HADOOP-13272.001.patch
        JIRA Issue HADOOP-13272
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 88166a50cb43 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 2ca7344
        Default Java 1.8.0_91
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/9794/artifact/patchprocess/diff-checkstyle-root.txt
        unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9794/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
        unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9794/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9794/testReport/
        modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: .
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9794/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 26s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files. 0 mvndep 0m 13s Maven dependency ordering for branch +1 mvninstall 8m 32s trunk passed +1 compile 7m 22s trunk passed +1 checkstyle 1m 24s trunk passed +1 mvnsite 1m 57s trunk passed +1 mvneclipse 0m 26s trunk passed +1 findbugs 3m 15s trunk passed +1 javadoc 1m 40s trunk passed 0 mvndep 0m 13s Maven dependency ordering for patch +1 mvninstall 1m 30s the patch passed +1 compile 7m 6s the patch passed +1 javac 7m 6s the patch passed -1 checkstyle 1m 24s root: The patch generated 7 new + 181 unchanged - 1 fixed = 188 total (was 182) +1 mvnsite 1m 49s the patch passed +1 mvneclipse 0m 29s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 3m 29s the patch passed +1 javadoc 1m 51s the patch passed -1 unit 7m 52s hadoop-common in the patch failed. -1 unit 65m 10s hadoop-hdfs in the patch failed. +1 asflicense 0m 23s The patch does not generate ASF License warnings. 117m 25s Reason Tests Failed junit tests hadoop.metrics2.impl.TestGangliaMetrics   hadoop.hdfs.server.blockmanagement.TestReplicationPolicy Subsystem Report/Notes Docker Image:yetus/hadoop:e2f6409 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12810976/HADOOP-13272.001.patch JIRA Issue HADOOP-13272 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 88166a50cb43 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 2ca7344 Default Java 1.8.0_91 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/9794/artifact/patchprocess/diff-checkstyle-root.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9794/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9794/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9794/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9794/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.

          People

          • Assignee:
            shihaoliang Peter Shi
            Reporter:
            shihaoliang Peter Shi
          • Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development