Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-8345

Storage policy APIs must be exposed via the FileSystem interface

    Details

    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      The storage policy APIs are not exposed via FileSystem. Since DistributedFileSystem is tagged as LimitedPrivate we should expose the APIs through FileSystem for use by other applications.

      1. HDFS-8345.01.patch
        12 kB
        Arpit Agarwal
      2. HDFS-8345.02.patch
        14 kB
        Arpit Agarwal
      3. HDFS-8345.03.patch
        19 kB
        Arpit Agarwal
      4. HDFS-8345.04.patch
        21 kB
        Arpit Agarwal
      5. HDFS-8345.05.patch
        21 kB
        Arpit Agarwal
      6. HDFS-8345.06.patch
        26 kB
        Arpit Agarwal
      7. HDFS-8345.07.patch
        26 kB
        Arpit Agarwal

        Issue Links

          Activity

          Hide
          xyao Xiaoyu Yao added a comment -

          The patch looks good to me. I just have few comments:

          1. NIT: Missing "@throws IOException" for Javadoc of FileSystem#setStoragePolicy().

          2. Do we miss the ViewFS#getStoragePolicyNames() override since ViewFS#setStoragePolicy is override. Or we expect it fallback to FileSystem#getStoragePoliciyNames that throws UnsupportedOperationException?

          3. Can you add a unit test for the new API getStoragePolicyNames()?

          Show
          xyao Xiaoyu Yao added a comment - The patch looks good to me. I just have few comments: 1. NIT: Missing "@throws IOException" for Javadoc of FileSystem#setStoragePolicy(). 2. Do we miss the ViewFS#getStoragePolicyNames() override since ViewFS#setStoragePolicy is override. Or we expect it fallback to FileSystem#getStoragePoliciyNames that throws UnsupportedOperationException? 3. Can you add a unit test for the new API getStoragePolicyNames()?
          Hide
          arpitagarwal Arpit Agarwal added a comment -

          Thanks Xiaoyu Yao. Added test and fixed missing exception in Javadoc.

          2. Do we miss the ViewFS#getStoragePolicyNames() override since ViewFS#setStoragePolicy is override. Or we expect it fallback to FileSystem#getStoragePoliciyNames that throws UnsupportedOperationException?

          Correct. In the absence of a path name we cannot know which NameSystem to forward the call.

          Show
          arpitagarwal Arpit Agarwal added a comment - Thanks Xiaoyu Yao . Added test and fixed missing exception in Javadoc. 2. Do we miss the ViewFS#getStoragePolicyNames() override since ViewFS#setStoragePolicy is override. Or we expect it fallback to FileSystem#getStoragePoliciyNames that throws UnsupportedOperationException? Correct. In the absence of a path name we cannot know which NameSystem to forward the call.
          Hide
          xyao Xiaoyu Yao added a comment -

          +1 for the v2 patch pending Jenkins.

          Show
          xyao Xiaoyu Yao added a comment - +1 for the v2 patch pending Jenkins.
          Hide
          jingzhao Jing Zhao added a comment -

          Thanks for working on this, Arpit! The patch looks good to me overall. My only concern is that only exposing setStoragePolicy and getStoragePoliciyNames may be incomplete, since the user will only know the names of the policies. Since StorageType is already in org.apache.hadoop.fs, how about also moving BlockStoragePolicy into org.apache.hadoop.fs and expose it as public? Then we can expose both setStoragePolicy and getStoragePolicies in FileSystem.

          Show
          jingzhao Jing Zhao added a comment - Thanks for working on this, Arpit! The patch looks good to me overall. My only concern is that only exposing setStoragePolicy and getStoragePoliciyNames may be incomplete, since the user will only know the names of the policies. Since StorageType is already in org.apache.hadoop.fs , how about also moving BlockStoragePolicy into org.apache.hadoop.fs and expose it as public? Then we can expose both setStoragePolicy and getStoragePolicies in FileSystem .
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 14m 37s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
          +1 javac 7m 29s There were no new javac warning messages.
          +1 javadoc 9m 37s There were no new javadoc warning messages.
          +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 1m 31s The applied patch generated 3 new checkstyle issues (total was 270, now 271).
          -1 whitespace 0m 1s The patch has 4 line(s) that end in whitespace. Use git apply --whitespace=fix.
          +1 install 1m 38s mvn install still works.
          +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse.
          +1 findbugs 4m 43s The patch does not introduce any new Findbugs (version 2.0.3) warnings.
          -1 common tests 22m 25s Tests failed in hadoop-common.
          -1 hdfs tests 165m 44s Tests failed in hadoop-hdfs.
              229m 8s  



          Reason Tests
          Failed unit tests hadoop.fs.TestFilterFileSystem
            hadoop.fs.TestHarFileSystem
            hadoop.tools.TestHdfsConfigFields
            hadoop.tracing.TestTraceAdmin



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12732420/HDFS-8345.02.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / f24452d
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/10941/artifact/patchprocess/diffcheckstylehadoop-common.txt
          whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/10941/artifact/patchprocess/whitespace.txt
          hadoop-common test log https://builds.apache.org/job/PreCommit-HDFS-Build/10941/artifact/patchprocess/testrun_hadoop-common.txt
          hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/10941/artifact/patchprocess/testrun_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/10941/testReport/
          Java 1.7.0_55
          uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/10941/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 14m 37s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 7m 29s There were no new javac warning messages. +1 javadoc 9m 37s There were no new javadoc warning messages. +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 1m 31s The applied patch generated 3 new checkstyle issues (total was 270, now 271). -1 whitespace 0m 1s The patch has 4 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 install 1m 38s mvn install still works. +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse. +1 findbugs 4m 43s The patch does not introduce any new Findbugs (version 2.0.3) warnings. -1 common tests 22m 25s Tests failed in hadoop-common. -1 hdfs tests 165m 44s Tests failed in hadoop-hdfs.     229m 8s   Reason Tests Failed unit tests hadoop.fs.TestFilterFileSystem   hadoop.fs.TestHarFileSystem   hadoop.tools.TestHdfsConfigFields   hadoop.tracing.TestTraceAdmin Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12732420/HDFS-8345.02.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / f24452d checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/10941/artifact/patchprocess/diffcheckstylehadoop-common.txt whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/10941/artifact/patchprocess/whitespace.txt hadoop-common test log https://builds.apache.org/job/PreCommit-HDFS-Build/10941/artifact/patchprocess/testrun_hadoop-common.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/10941/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/10941/testReport/ Java 1.7.0_55 uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/10941/console This message was automatically generated.
          Hide
          arpitagarwal Arpit Agarwal added a comment -

          Thanks for the review Jing Zhao. I wanted to do so but wasn't sure we want to expose BlockStoragePolicy publicly. There are too many public functions that should be internal to HDFS. What do you think?

          Show
          arpitagarwal Arpit Agarwal added a comment - Thanks for the review Jing Zhao . I wanted to do so but wasn't sure we want to expose BlockStoragePolicy publicly. There are too many public functions that should be internal to HDFS. What do you think?
          Hide
          jingzhao Jing Zhao added a comment -

          Looks like most of the public functions in BlockStoragePolicy are generically choosing StorageType for replicas, and replication is a general concept and is already included in FileStatus. If you think some functions should be internal to HDFS, maybe we can also have a new subclass/wrapper of BlockStoragePolicy in HDFS?

          Show
          jingzhao Jing Zhao added a comment - Looks like most of the public functions in BlockStoragePolicy are generically choosing StorageType for replicas, and replication is a general concept and is already included in FileStatus . If you think some functions should be internal to HDFS, maybe we can also have a new subclass/wrapper of BlockStoragePolicy in HDFS?
          Hide
          arpitagarwal Arpit Agarwal added a comment -

          On further thought I'll have the base class return a list of BlockStoragePolicySpi which exposes the policy details.

          BlockStoragePolicy can implement BlockStoragePolicySpi. Will post an updated patch shortly.

          Show
          arpitagarwal Arpit Agarwal added a comment - On further thought I'll have the base class return a list of BlockStoragePolicySpi which exposes the policy details. BlockStoragePolicy can implement BlockStoragePolicySpi. Will post an updated patch shortly.
          Hide
          arpitagarwal Arpit Agarwal added a comment -

          Thank you Jing. I didn't see your latest comment last time.

          The updated patch does something similar, it adds an interface BlockStoragePolicySpi which can be used to query all storage policy properties of interest to applications. BlockStoragePolicy inherits the interface and remains private.

          Also fixed a checkstyle issue, the remaining ones are unrelated to the patch.

          Show
          arpitagarwal Arpit Agarwal added a comment - Thank you Jing. I didn't see your latest comment last time. The updated patch does something similar, it adds an interface BlockStoragePolicySpi which can be used to query all storage policy properties of interest to applications. BlockStoragePolicy inherits the interface and remains private. Also fixed a checkstyle issue, the remaining ones are unrelated to the patch.
          Hide
          arpitagarwal Arpit Agarwal added a comment -

          Also I added a new call FileSystem#getAllStoragePolicies.

          I left DistributedFileSystem#getStoragePolicies as it is and tagged it @Deprecated.

          Show
          arpitagarwal Arpit Agarwal added a comment - Also I added a new call FileSystem#getAllStoragePolicies . I left DistributedFileSystem#getStoragePolicies as it is and tagged it @Deprecated .
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 14m 58s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
          -1 javac 7m 41s The applied patch generated 4 additional warning messages.
          -1 javadoc 9m 52s The applied patch generated 3 additional warning messages.
          +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 2m 27s The applied patch generated 3 new checkstyle issues (total was 269, now 270).
          -1 whitespace 0m 2s The patch has 2 line(s) that end in whitespace. Use git apply --whitespace=fix.
          +1 install 1m 38s mvn install still works.
          +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse.
          +1 findbugs 5m 26s The patch does not introduce any new Findbugs (version 2.0.3) warnings.
          -1 common tests 22m 38s Tests failed in hadoop-common.
          -1 hdfs tests 140m 40s Tests failed in hadoop-hdfs.
          -1 hdfs tests 0m 14s Tests failed in hadoop-hdfs-client.
              207m 0s  



          Reason Tests
          Failed unit tests hadoop.fs.TestFilterFileSystem
            hadoop.fs.TestHarFileSystem
          Timed out tests org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistFiles
          Failed build hadoop-hdfs-client



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12732656/HDFS-8345.03.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 93b770f
          javac https://builds.apache.org/job/PreCommit-HDFS-Build/10956/artifact/patchprocess/diffJavacWarnings.txt
          javadoc https://builds.apache.org/job/PreCommit-HDFS-Build/10956/artifact/patchprocess/diffJavadocWarnings.txt
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/10956/artifact/patchprocess/diffcheckstylehadoop-common.txt
          whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/10956/artifact/patchprocess/whitespace.txt
          hadoop-common test log https://builds.apache.org/job/PreCommit-HDFS-Build/10956/artifact/patchprocess/testrun_hadoop-common.txt
          hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/10956/artifact/patchprocess/testrun_hadoop-hdfs.txt
          hadoop-hdfs-client test log https://builds.apache.org/job/PreCommit-HDFS-Build/10956/artifact/patchprocess/testrun_hadoop-hdfs-client.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/10956/testReport/
          Java 1.7.0_55
          uname Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/10956/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 14m 58s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. -1 javac 7m 41s The applied patch generated 4 additional warning messages. -1 javadoc 9m 52s The applied patch generated 3 additional warning messages. +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 2m 27s The applied patch generated 3 new checkstyle issues (total was 269, now 270). -1 whitespace 0m 2s The patch has 2 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 install 1m 38s mvn install still works. +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse. +1 findbugs 5m 26s The patch does not introduce any new Findbugs (version 2.0.3) warnings. -1 common tests 22m 38s Tests failed in hadoop-common. -1 hdfs tests 140m 40s Tests failed in hadoop-hdfs. -1 hdfs tests 0m 14s Tests failed in hadoop-hdfs-client.     207m 0s   Reason Tests Failed unit tests hadoop.fs.TestFilterFileSystem   hadoop.fs.TestHarFileSystem Timed out tests org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistFiles Failed build hadoop-hdfs-client Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12732656/HDFS-8345.03.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 93b770f javac https://builds.apache.org/job/PreCommit-HDFS-Build/10956/artifact/patchprocess/diffJavacWarnings.txt javadoc https://builds.apache.org/job/PreCommit-HDFS-Build/10956/artifact/patchprocess/diffJavadocWarnings.txt checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/10956/artifact/patchprocess/diffcheckstylehadoop-common.txt whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/10956/artifact/patchprocess/whitespace.txt hadoop-common test log https://builds.apache.org/job/PreCommit-HDFS-Build/10956/artifact/patchprocess/testrun_hadoop-common.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/10956/artifact/patchprocess/testrun_hadoop-hdfs.txt hadoop-hdfs-client test log https://builds.apache.org/job/PreCommit-HDFS-Build/10956/artifact/patchprocess/testrun_hadoop-hdfs-client.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/10956/testReport/ Java 1.7.0_55 uname Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/10956/console This message was automatically generated.
          Hide
          arpitagarwal Arpit Agarwal added a comment -

          Fixed failed tests. Remaining Jenkins issues look like false positives.

          Show
          arpitagarwal Arpit Agarwal added a comment - Fixed failed tests. Remaining Jenkins issues look like false positives.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          -1 patch 0m 0s The patch command could not apply the patch during dryrun.



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12732978/HDFS-8345.04.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 09fe16f
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/10989/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 patch 0m 0s The patch command could not apply the patch during dryrun. Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12732978/HDFS-8345.04.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 09fe16f Console output https://builds.apache.org/job/PreCommit-HDFS-Build/10989/console This message was automatically generated.
          Hide
          arpitagarwal Arpit Agarwal added a comment -

          v5: Rebase to trunk.

          Show
          arpitagarwal Arpit Agarwal added a comment - v5: Rebase to trunk.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 15m 11s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 2 new or modified test files.
          -1 javac 7m 39s The applied patch generated 4 additional warning messages.
          -1 javadoc 9m 50s The applied patch generated 3 additional warning messages.
          +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 2m 30s The applied patch generated 3 new checkstyle issues (total was 286, now 287).
          -1 whitespace 0m 2s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix.
          +1 install 1m 39s mvn install still works.
          +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
          +1 findbugs 5m 31s The patch does not introduce any new Findbugs (version 2.0.3) warnings.
          +1 common tests 23m 15s Tests passed in hadoop-common.
          -1 hdfs tests 167m 59s Tests failed in hadoop-hdfs.
          +1 hdfs tests 0m 16s Tests passed in hadoop-hdfs-client.
              235m 19s  



          Reason Tests
          Failed unit tests hadoop.tools.TestHdfsConfigFields



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12732993/HDFS-8345.05.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 9a2a955
          javac https://builds.apache.org/job/PreCommit-HDFS-Build/10993/artifact/patchprocess/diffJavacWarnings.txt
          javadoc https://builds.apache.org/job/PreCommit-HDFS-Build/10993/artifact/patchprocess/diffJavadocWarnings.txt
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/10993/artifact/patchprocess/diffcheckstylehadoop-common.txt
          whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/10993/artifact/patchprocess/whitespace.txt
          hadoop-common test log https://builds.apache.org/job/PreCommit-HDFS-Build/10993/artifact/patchprocess/testrun_hadoop-common.txt
          hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/10993/artifact/patchprocess/testrun_hadoop-hdfs.txt
          hadoop-hdfs-client test log https://builds.apache.org/job/PreCommit-HDFS-Build/10993/artifact/patchprocess/testrun_hadoop-hdfs-client.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/10993/testReport/
          Java 1.7.0_55
          uname Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/10993/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 15m 11s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 2 new or modified test files. -1 javac 7m 39s The applied patch generated 4 additional warning messages. -1 javadoc 9m 50s The applied patch generated 3 additional warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 2m 30s The applied patch generated 3 new checkstyle issues (total was 286, now 287). -1 whitespace 0m 2s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 install 1m 39s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 5m 31s The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 common tests 23m 15s Tests passed in hadoop-common. -1 hdfs tests 167m 59s Tests failed in hadoop-hdfs. +1 hdfs tests 0m 16s Tests passed in hadoop-hdfs-client.     235m 19s   Reason Tests Failed unit tests hadoop.tools.TestHdfsConfigFields Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12732993/HDFS-8345.05.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 9a2a955 javac https://builds.apache.org/job/PreCommit-HDFS-Build/10993/artifact/patchprocess/diffJavacWarnings.txt javadoc https://builds.apache.org/job/PreCommit-HDFS-Build/10993/artifact/patchprocess/diffJavadocWarnings.txt checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/10993/artifact/patchprocess/diffcheckstylehadoop-common.txt whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/10993/artifact/patchprocess/whitespace.txt hadoop-common test log https://builds.apache.org/job/PreCommit-HDFS-Build/10993/artifact/patchprocess/testrun_hadoop-common.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/10993/artifact/patchprocess/testrun_hadoop-hdfs.txt hadoop-hdfs-client test log https://builds.apache.org/job/PreCommit-HDFS-Build/10993/artifact/patchprocess/testrun_hadoop-hdfs-client.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/10993/testReport/ Java 1.7.0_55 uname Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/10993/console This message was automatically generated.
          Hide
          jingzhao Jing Zhao added a comment -

          Thanks for updating the patch, Arpit! The latest patch looks pretty good to me. But looks like still need to fix the javac/javadoc warnings. Apart from that +1.

          Show
          jingzhao Jing Zhao added a comment - Thanks for updating the patch, Arpit! The latest patch looks pretty good to me. But looks like still need to fix the javac/javadoc warnings. Apart from that +1.
          Hide
          busbey Sean Busbey added a comment -

          It would be nice if the returned values from getAllStoragePolicies were usable in a setStoragePolicy method.

          Could we update the FileSystem specification documents?

          +   * Set the storage policy for a given file or directory.
          +   *
          +   * @param path file or directory path.
          +   * @param policyName the name of the target storage policy. The list
          +   *                   of supported Storage policies can be retrieved
          +   *                   via {@link #getStoragePolicyNames}.
          +   */
          +  public void setStoragePolicy(final Path path, final String policyName)
          +      throws IOException {
          

          Note that it's an optional operation and some implementations will throw UnsupportOperationException.

          +  /**
          +   * Retrieve all the storage policies supported by this file system.
          +   *
          +   * @return all storage policies supported by this filesystem.
          +   * @throws IOException
          +   */
          +  public List<? extends BlockStoragePolicySpi> getAllStoragePolicies()
          +      throws IOException {
          

          Note that it's an optional operation and some implementations will throw UnsupportOperationException.

          +   * @param policyName the name of the target storage policy. The list
          +   *                   of supported Storage policies can be retrieved
          +   *                   via {@link #getStoragePolicyNames}.
          +   */
          

          The method "getStoragePolicyNames" didn't make it into the patch. Update to point at new method?

          +  public List<? extends BlockStoragePolicySpi> getAllStoragePolicies()
          +      throws IOException {
          

          Should this return type be Collection<? extends BlockStoragePolicySpi> ? does the ordering matter? Maybe it should be a Set?

          +public interface BlockStoragePolicySpi {
          +
          

          Could we switch to calling the interface BlockStoragePolicy and make the one in HDFS "DFSBlockStoragePolicy" or the like?
          "Spi" is a term of art that's going to be less accessible for folks.

          
          +  StorageType[] getStorageTypes();
          

          Should the various return types in BlockStoragePolicySpi be returning List or Set or SortedSet?

               return fs.createNonRecursive(f, permission, flags, bufferSize, replication, blockSize,
          -        progress);
          +                                 progress);
             }
          

          nit: this whitespace fix doesn't appear related to the rest of the changes in the file (but I'm not sure what Hadoop's norm is for 'related')

          Show
          busbey Sean Busbey added a comment - It would be nice if the returned values from getAllStoragePolicies were usable in a setStoragePolicy method. Could we update the FileSystem specification documents? + * Set the storage policy for a given file or directory. + * + * @param path file or directory path. + * @param policyName the name of the target storage policy. The list + * of supported Storage policies can be retrieved + * via {@link #getStoragePolicyNames}. + */ + public void setStoragePolicy( final Path path, final String policyName) + throws IOException { Note that it's an optional operation and some implementations will throw UnsupportOperationException. + /** + * Retrieve all the storage policies supported by this file system. + * + * @ return all storage policies supported by this filesystem. + * @ throws IOException + */ + public List<? extends BlockStoragePolicySpi> getAllStoragePolicies() + throws IOException { Note that it's an optional operation and some implementations will throw UnsupportOperationException. + * @param policyName the name of the target storage policy. The list + * of supported Storage policies can be retrieved + * via {@link #getStoragePolicyNames}. + */ The method "getStoragePolicyNames" didn't make it into the patch. Update to point at new method? + public List<? extends BlockStoragePolicySpi> getAllStoragePolicies() + throws IOException { Should this return type be Collection<? extends BlockStoragePolicySpi> ? does the ordering matter? Maybe it should be a Set? + public interface BlockStoragePolicySpi { + Could we switch to calling the interface BlockStoragePolicy and make the one in HDFS "DFSBlockStoragePolicy" or the like? "Spi" is a term of art that's going to be less accessible for folks. + StorageType[] getStorageTypes(); Should the various return types in BlockStoragePolicySpi be returning List or Set or SortedSet? return fs.createNonRecursive(f, permission, flags, bufferSize, replication, blockSize, - progress); + progress); } nit: this whitespace fix doesn't appear related to the rest of the changes in the file (but I'm not sure what Hadoop's norm is for 'related')
          Hide
          arpitagarwal Arpit Agarwal added a comment -

          Updated patch to fix the warnings due to the newly deprecated API.

          Show
          arpitagarwal Arpit Agarwal added a comment - Updated patch to fix the warnings due to the newly deprecated API.
          Hide
          arpitagarwal Arpit Agarwal added a comment - - edited

          Thank you for the detailed review Sean Busbey.

          It would be nice if the returned values from getAllStoragePolicies were usable in a setStoragePolicy method.

          getAllStoragePolicies returns a collection, you can use BlockStoragePolicySpi#getName on a collection element as the parameter to setStoragePolicy.

          Could we update the FileSystem specification documents?

          Which specification docs are you referring to?

          Note that it's an optional operation and some implementations will throw UnsupportOperationException.

          Let's do that separately for all FileSystem API calls that fit the same pattern.

          The method "getStoragePolicyNames" didn't make it into the patch. Update to point at new method?

          Fixed.

          Should this return type be Collection<? extends BlockStoragePolicySpi> ? does the ordering matter? Maybe it should be a Set?

          Fixed to return a Collection.

          Could we switch to calling the interface BlockStoragePolicy and make the one in HDFS "DFSBlockStoragePolicy" or the like? "Spi" is a term of art that's going to be less accessible for folks.

          Should the various return types in BlockStoragePolicySpi be returning List or Set or SortedSet?

          I agree on both points. However it's likely incompatible since BlockStoragePolicy was exposed via a limited private interface in 2.6.

          nit: this whitespace fix doesn't appear related to the rest of the changes in the file (but I'm not sure what Hadoop's norm is for 'related')

          Good catch, IntelliJ slipped that in. Fixed.

          Show
          arpitagarwal Arpit Agarwal added a comment - - edited Thank you for the detailed review Sean Busbey . It would be nice if the returned values from getAllStoragePolicies were usable in a setStoragePolicy method. getAllStoragePolicies returns a collection, you can use BlockStoragePolicySpi#getName on a collection element as the parameter to setStoragePolicy. Could we update the FileSystem specification documents? Which specification docs are you referring to? Note that it's an optional operation and some implementations will throw UnsupportOperationException. Let's do that separately for all FileSystem API calls that fit the same pattern. The method "getStoragePolicyNames" didn't make it into the patch. Update to point at new method? Fixed. Should this return type be Collection<? extends BlockStoragePolicySpi> ? does the ordering matter? Maybe it should be a Set? Fixed to return a Collection. Could we switch to calling the interface BlockStoragePolicy and make the one in HDFS "DFSBlockStoragePolicy" or the like? "Spi" is a term of art that's going to be less accessible for folks. Should the various return types in BlockStoragePolicySpi be returning List or Set or SortedSet? I agree on both points. However it's likely incompatible since BlockStoragePolicy was exposed via a limited private interface in 2.6. nit: this whitespace fix doesn't appear related to the rest of the changes in the file (but I'm not sure what Hadoop's norm is for 'related') Good catch, IntelliJ slipped that in. Fixed.
          Hide
          busbey Sean Busbey added a comment -

          It would be nice if the returned values from getAllStoragePolicies were usable in a setStoragePolicy method.

          getAllStoragePolicies returns a collection, you can use BlockStoragePolicySpi#getName on a collection element as the parameter to setStoragePolicy.

          That's not so bad. Is there a reason not to include a convenience method that just takes a BlockStoragePolicySpi object directly?

          Could we update the FileSystem specification documents?

          Which specification docs are you referring to?

          The one in the docs: http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/filesystem/index.html

          Could we switch to calling the interface BlockStoragePolicy and make the one in HDFS "DFSBlockStoragePolicy" or the like? "Spi" is a term of art that's going to be less accessible for folks.

          Should the various return types in BlockStoragePolicySpi be returning List or Set or SortedSet?

          I agree on both points. However it's likely incompatible since BlockStoragePolicy was exposed via a limited private interface in 2.6.

          Can you point me to the exposure? BlockStoragePolicy is IA.Private, and I would expect we can use covariant return types to make a source-compatible update.

          Show
          busbey Sean Busbey added a comment - It would be nice if the returned values from getAllStoragePolicies were usable in a setStoragePolicy method. getAllStoragePolicies returns a collection, you can use BlockStoragePolicySpi#getName on a collection element as the parameter to setStoragePolicy. That's not so bad. Is there a reason not to include a convenience method that just takes a BlockStoragePolicySpi object directly? Could we update the FileSystem specification documents? Which specification docs are you referring to? The one in the docs: http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/filesystem/index.html Could we switch to calling the interface BlockStoragePolicy and make the one in HDFS "DFSBlockStoragePolicy" or the like? "Spi" is a term of art that's going to be less accessible for folks. Should the various return types in BlockStoragePolicySpi be returning List or Set or SortedSet? I agree on both points. However it's likely incompatible since BlockStoragePolicy was exposed via a limited private interface in 2.6. Can you point me to the exposure? BlockStoragePolicy is IA.Private, and I would expect we can use covariant return types to make a source-compatible update.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 14m 31s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 2 new or modified test files.
          +1 javac 7m 28s There were no new javac warning messages.
          -1 javadoc 9m 32s The applied patch generated 2 additional warning messages.
          +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 2m 23s The applied patch generated 2 new checkstyle issues (total was 286, now 286).
          +1 whitespace 0m 2s The patch has no lines that end in whitespace.
          +1 install 1m 36s mvn install still works.
          +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
          +1 findbugs 5m 25s The patch does not introduce any new Findbugs (version 2.0.3) warnings.
          +1 common tests 23m 36s Tests passed in hadoop-common.
          +1 hdfs tests 168m 7s Tests passed in hadoop-hdfs.
          +1 hdfs tests 0m 16s Tests passed in hadoop-hdfs-client.
              234m 22s  



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12733187/HDFS-8345.06.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 03a293a
          javadoc https://builds.apache.org/job/PreCommit-HDFS-Build/11006/artifact/patchprocess/diffJavadocWarnings.txt
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/11006/artifact/patchprocess/diffcheckstylehadoop-common.txt
          hadoop-common test log https://builds.apache.org/job/PreCommit-HDFS-Build/11006/artifact/patchprocess/testrun_hadoop-common.txt
          hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/11006/artifact/patchprocess/testrun_hadoop-hdfs.txt
          hadoop-hdfs-client test log https://builds.apache.org/job/PreCommit-HDFS-Build/11006/artifact/patchprocess/testrun_hadoop-hdfs-client.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11006/testReport/
          Java 1.7.0_55
          uname Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11006/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 14m 31s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 2 new or modified test files. +1 javac 7m 28s There were no new javac warning messages. -1 javadoc 9m 32s The applied patch generated 2 additional warning messages. +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 2m 23s The applied patch generated 2 new checkstyle issues (total was 286, now 286). +1 whitespace 0m 2s The patch has no lines that end in whitespace. +1 install 1m 36s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. +1 findbugs 5m 25s The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 common tests 23m 36s Tests passed in hadoop-common. +1 hdfs tests 168m 7s Tests passed in hadoop-hdfs. +1 hdfs tests 0m 16s Tests passed in hadoop-hdfs-client.     234m 22s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12733187/HDFS-8345.06.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 03a293a javadoc https://builds.apache.org/job/PreCommit-HDFS-Build/11006/artifact/patchprocess/diffJavadocWarnings.txt checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/11006/artifact/patchprocess/diffcheckstylehadoop-common.txt hadoop-common test log https://builds.apache.org/job/PreCommit-HDFS-Build/11006/artifact/patchprocess/testrun_hadoop-common.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/11006/artifact/patchprocess/testrun_hadoop-hdfs.txt hadoop-hdfs-client test log https://builds.apache.org/job/PreCommit-HDFS-Build/11006/artifact/patchprocess/testrun_hadoop-hdfs-client.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11006/testReport/ Java 1.7.0_55 uname Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11006/console This message was automatically generated.
          Hide
          arpitagarwal Arpit Agarwal added a comment -

          That's not so bad. Is there a reason not to include a convenience method that just takes a BlockStoragePolicySpi object directly?

          That doesn't sound useful enough to introduce one more FileSystem API. Can you think of an example where the extra API call would simplify application code?

          Can you point me to the exposure? BlockStoragePolicy is IA.Private, and I would expect we can use covariant return types to make a source-compatible update.

          It was tagged IA.Private but returned via a LimitedPrivate interface, unfortunate as that is. Changing the name, package or public interface of BlockStoragePolicy would technically be incompatible. However if you suggest a better name for the new interface I'd be happy to change it.

          The one in the docs: http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/filesystem/index.html

          Okay let me file a separate Jira for that.

          Show
          arpitagarwal Arpit Agarwal added a comment - That's not so bad. Is there a reason not to include a convenience method that just takes a BlockStoragePolicySpi object directly? That doesn't sound useful enough to introduce one more FileSystem API. Can you think of an example where the extra API call would simplify application code? Can you point me to the exposure? BlockStoragePolicy is IA.Private, and I would expect we can use covariant return types to make a source-compatible update. It was tagged IA.Private but returned via a LimitedPrivate interface, unfortunate as that is. Changing the name, package or public interface of BlockStoragePolicy would technically be incompatible. However if you suggest a better name for the new interface I'd be happy to change it. The one in the docs: http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/filesystem/index.html Okay let me file a separate Jira for that.
          Hide
          arpitagarwal Arpit Agarwal added a comment -

          v7 to fix Javadoc missed last time.

          Show
          arpitagarwal Arpit Agarwal added a comment - v7 to fix Javadoc missed last time.
          Hide
          jingzhao Jing Zhao added a comment -

          Is there a reason not to include a convenience method that just takes a BlockStoragePolicySpi object directly

          Currently we still expect admins to set up policies first, and the users later use names of existing policies to config files/directories. Directly using a policy object in the API will allow users to pass self-created policies and also bring in extra complexity for server side verification.

          +1 on the 07 patch.

          Show
          jingzhao Jing Zhao added a comment - Is there a reason not to include a convenience method that just takes a BlockStoragePolicySpi object directly Currently we still expect admins to set up policies first, and the users later use names of existing policies to config files/directories. Directly using a policy object in the API will allow users to pass self-created policies and also bring in extra complexity for server side verification. +1 on the 07 patch.
          Hide
          busbey Sean Busbey added a comment -

          Can you point me to the exposure? BlockStoragePolicy is IA.Private, and I would expect we can use covariant return types to make a source-compatible update.

          It was tagged IA.Private but returned via a LimitedPrivate interface, unfortunate as that is. Changing the name, package or public interface of BlockStoragePolicy would technically be incompatible. However if you suggest a better name for the new interface I'd be happy to change it.

          Which LimitedPrivate though? I'd like to make sure we can't do the change in a compatible way.

          Show
          busbey Sean Busbey added a comment - Can you point me to the exposure? BlockStoragePolicy is IA.Private, and I would expect we can use covariant return types to make a source-compatible update. It was tagged IA.Private but returned via a LimitedPrivate interface, unfortunate as that is. Changing the name, package or public interface of BlockStoragePolicy would technically be incompatible. However if you suggest a better name for the new interface I'd be happy to change it. Which LimitedPrivate though? I'd like to make sure we can't do the change in a compatible way.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 15m 17s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 2 new or modified test files.
          +1 javac 7m 44s There were no new javac warning messages.
          +1 javadoc 10m 0s There were no new javadoc warning messages.
          +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 2m 26s The applied patch generated 3 new checkstyle issues (total was 286, now 287).
          +1 whitespace 0m 2s The patch has no lines that end in whitespace.
          +1 install 1m 37s mvn install still works.
          +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse.
          -1 findbugs 2m 5s The patch appears to cause Findbugs (version 2.0.3) to fail.
          +1 common tests 23m 5s Tests passed in hadoop-common.
          -1 hdfs tests 0m 30s Tests failed in hadoop-hdfs.
          +1 hdfs tests 0m 15s Tests passed in hadoop-hdfs-client.
              64m 25s  



          Reason Tests
          Failed build hadoop-hdfs



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12733254/HDFS-8345.07.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / f7e051c
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/11014/artifact/patchprocess/diffcheckstylehadoop-common.txt
          hadoop-common test log https://builds.apache.org/job/PreCommit-HDFS-Build/11014/artifact/patchprocess/testrun_hadoop-common.txt
          hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/11014/artifact/patchprocess/testrun_hadoop-hdfs.txt
          hadoop-hdfs-client test log https://builds.apache.org/job/PreCommit-HDFS-Build/11014/artifact/patchprocess/testrun_hadoop-hdfs-client.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11014/testReport/
          Java 1.7.0_55
          uname Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11014/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 15m 17s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 2 new or modified test files. +1 javac 7m 44s There were no new javac warning messages. +1 javadoc 10m 0s There were no new javadoc warning messages. +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 2m 26s The applied patch generated 3 new checkstyle issues (total was 286, now 287). +1 whitespace 0m 2s The patch has no lines that end in whitespace. +1 install 1m 37s mvn install still works. +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse. -1 findbugs 2m 5s The patch appears to cause Findbugs (version 2.0.3) to fail. +1 common tests 23m 5s Tests passed in hadoop-common. -1 hdfs tests 0m 30s Tests failed in hadoop-hdfs. +1 hdfs tests 0m 15s Tests passed in hadoop-hdfs-client.     64m 25s   Reason Tests Failed build hadoop-hdfs Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12733254/HDFS-8345.07.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / f7e051c checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/11014/artifact/patchprocess/diffcheckstylehadoop-common.txt hadoop-common test log https://builds.apache.org/job/PreCommit-HDFS-Build/11014/artifact/patchprocess/testrun_hadoop-common.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/11014/artifact/patchprocess/testrun_hadoop-hdfs.txt hadoop-hdfs-client test log https://builds.apache.org/job/PreCommit-HDFS-Build/11014/artifact/patchprocess/testrun_hadoop-hdfs-client.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11014/testReport/ Java 1.7.0_55 uname Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11014/console This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 14m 34s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 2 new or modified test files.
          +1 javac 7m 30s There were no new javac warning messages.
          +1 javadoc 9m 42s There were no new javadoc warning messages.
          +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 2m 24s The applied patch generated 2 new checkstyle issues (total was 286, now 286).
          +1 whitespace 0m 2s The patch has no lines that end in whitespace.
          +1 install 1m 37s mvn install still works.
          +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse.
          +1 findbugs 5m 22s The patch does not introduce any new Findbugs (version 2.0.3) warnings.
          +1 common tests 22m 37s Tests passed in hadoop-common.
          -1 hdfs tests 168m 35s Tests failed in hadoop-hdfs.
          -1 hdfs tests 0m 16s Tests failed in hadoop-hdfs-client.
              234m 3s  



          Reason Tests
          Failed build hadoop-hdfs
            hadoop-hdfs-client



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12733254/HDFS-8345.07.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / f7e051c
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/11016/artifact/patchprocess/diffcheckstylehadoop-common.txt
          hadoop-common test log https://builds.apache.org/job/PreCommit-HDFS-Build/11016/artifact/patchprocess/testrun_hadoop-common.txt
          hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/11016/artifact/patchprocess/testrun_hadoop-hdfs.txt
          hadoop-hdfs-client test log https://builds.apache.org/job/PreCommit-HDFS-Build/11016/artifact/patchprocess/testrun_hadoop-hdfs-client.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11016/testReport/
          Java 1.7.0_55
          uname Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11016/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 14m 34s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 2 new or modified test files. +1 javac 7m 30s There were no new javac warning messages. +1 javadoc 9m 42s There were no new javadoc warning messages. +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 2m 24s The applied patch generated 2 new checkstyle issues (total was 286, now 286). +1 whitespace 0m 2s The patch has no lines that end in whitespace. +1 install 1m 37s mvn install still works. +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse. +1 findbugs 5m 22s The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 common tests 22m 37s Tests passed in hadoop-common. -1 hdfs tests 168m 35s Tests failed in hadoop-hdfs. -1 hdfs tests 0m 16s Tests failed in hadoop-hdfs-client.     234m 3s   Reason Tests Failed build hadoop-hdfs   hadoop-hdfs-client Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12733254/HDFS-8345.07.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / f7e051c checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/11016/artifact/patchprocess/diffcheckstylehadoop-common.txt hadoop-common test log https://builds.apache.org/job/PreCommit-HDFS-Build/11016/artifact/patchprocess/testrun_hadoop-common.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/11016/artifact/patchprocess/testrun_hadoop-hdfs.txt hadoop-hdfs-client test log https://builds.apache.org/job/PreCommit-HDFS-Build/11016/artifact/patchprocess/testrun_hadoop-hdfs-client.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11016/testReport/ Java 1.7.0_55 uname Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11016/console This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 14m 43s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 2 new or modified test files.
          +1 javac 7m 31s There were no new javac warning messages.
          +1 javadoc 9m 40s There were no new javadoc warning messages.
          +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 1m 54s The applied patch generated 2 new checkstyle issues (total was 286, now 286).
          +1 whitespace 0m 2s The patch has no lines that end in whitespace.
          +1 install 1m 36s mvn install still works.
          +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse.
          +1 findbugs 5m 26s The patch does not introduce any new Findbugs (version 2.0.3) warnings.
          +1 common tests 23m 42s Tests passed in hadoop-common.
          +1 hdfs tests 164m 26s Tests passed in hadoop-hdfs.
          +1 hdfs tests 0m 16s Tests passed in hadoop-hdfs-client.
              230m 58s  



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12733254/HDFS-8345.07.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / e453989
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/11017/artifact/patchprocess/diffcheckstylehadoop-common.txt
          hadoop-common test log https://builds.apache.org/job/PreCommit-HDFS-Build/11017/artifact/patchprocess/testrun_hadoop-common.txt
          hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/11017/artifact/patchprocess/testrun_hadoop-hdfs.txt
          hadoop-hdfs-client test log https://builds.apache.org/job/PreCommit-HDFS-Build/11017/artifact/patchprocess/testrun_hadoop-hdfs-client.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11017/testReport/
          Java 1.7.0_55
          uname Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11017/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 14m 43s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 2 new or modified test files. +1 javac 7m 31s There were no new javac warning messages. +1 javadoc 9m 40s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 1m 54s The applied patch generated 2 new checkstyle issues (total was 286, now 286). +1 whitespace 0m 2s The patch has no lines that end in whitespace. +1 install 1m 36s mvn install still works. +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse. +1 findbugs 5m 26s The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 common tests 23m 42s Tests passed in hadoop-common. +1 hdfs tests 164m 26s Tests passed in hadoop-hdfs. +1 hdfs tests 0m 16s Tests passed in hadoop-hdfs-client.     230m 58s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12733254/HDFS-8345.07.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / e453989 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/11017/artifact/patchprocess/diffcheckstylehadoop-common.txt hadoop-common test log https://builds.apache.org/job/PreCommit-HDFS-Build/11017/artifact/patchprocess/testrun_hadoop-common.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/11017/artifact/patchprocess/testrun_hadoop-hdfs.txt hadoop-hdfs-client test log https://builds.apache.org/job/PreCommit-HDFS-Build/11017/artifact/patchprocess/testrun_hadoop-hdfs-client.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11017/testReport/ Java 1.7.0_55 uname Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11017/console This message was automatically generated.
          Hide
          arpitagarwal Arpit Agarwal added a comment - - edited

          Hi Sean, it's returned via DistributedFileSystem (it's in the Jira description).

          Applications are using the storage policy APIs so we should avoid a breaking change unless necessary. Reclaiming the interface name is not enough reason.

          The checkstyle issues are not related to the patch. I will commit this shortly.

          Show
          arpitagarwal Arpit Agarwal added a comment - - edited Hi Sean, it's returned via DistributedFileSystem (it's in the Jira description). Applications are using the storage policy APIs so we should avoid a breaking change unless necessary. Reclaiming the interface name is not enough reason. The checkstyle issues are not related to the patch. I will commit this shortly.
          Hide
          arpitagarwal Arpit Agarwal added a comment -

          I have committed this for 2.8.0. Thank you for the reviews Xiaoyu Yao, Jing Zhao and Sean Busbey.

          Filed HADOOP-11981 to update the site docs.

          Show
          arpitagarwal Arpit Agarwal added a comment - I have committed this for 2.8.0. Thank you for the reviews Xiaoyu Yao , Jing Zhao and Sean Busbey . Filed HADOOP-11981 to update the site docs.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #7855 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7855/)
          HDFS-8345. Storage policy APIs must be exposed via the FileSystem interface. (Arpit Agarwal) (arp: rev a2190bf15d25e01fb4b220ba6401ce2f787a5c61)

          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ChRootedFs.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileContext.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFs.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestHarFileSystem.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockStoragePolicy.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/StoragePolicyAdmin.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/BlockStoragePolicySpi.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFileSystem.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/mover/Mover.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/AbstractFileSystem.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ViewFs.java
          • hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/BlockStoragePolicy.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/fs/Hdfs.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #7855 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7855/ ) HDFS-8345 . Storage policy APIs must be exposed via the FileSystem interface. (Arpit Agarwal) (arp: rev a2190bf15d25e01fb4b220ba6401ce2f787a5c61) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ChRootedFs.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileContext.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFs.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestHarFileSystem.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockStoragePolicy.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/StoragePolicyAdmin.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/BlockStoragePolicySpi.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFileSystem.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/mover/Mover.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/AbstractFileSystem.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ViewFs.java hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/BlockStoragePolicy.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/fs/Hdfs.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk #932 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/932/)
          HDFS-8345. Storage policy APIs must be exposed via the FileSystem interface. (Arpit Agarwal) (arp: rev a2190bf15d25e01fb4b220ba6401ce2f787a5c61)

          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/AbstractFileSystem.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ViewFs.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/fs/Hdfs.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockStoragePolicy.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/BlockStoragePolicySpi.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestHarFileSystem.java
          • hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/BlockStoragePolicy.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/StoragePolicyAdmin.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileContext.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFileSystem.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/mover/Mover.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFs.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ChRootedFs.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #932 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/932/ ) HDFS-8345 . Storage policy APIs must be exposed via the FileSystem interface. (Arpit Agarwal) (arp: rev a2190bf15d25e01fb4b220ba6401ce2f787a5c61) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/AbstractFileSystem.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ViewFs.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/fs/Hdfs.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockStoragePolicy.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/BlockStoragePolicySpi.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestHarFileSystem.java hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/BlockStoragePolicy.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/StoragePolicyAdmin.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileContext.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFileSystem.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/mover/Mover.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFs.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ChRootedFs.java
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Yarn-trunk-Java8 #201 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/201/)
          HDFS-8345. Storage policy APIs must be exposed via the FileSystem interface. (Arpit Agarwal) (arp: rev a2190bf15d25e01fb4b220ba6401ce2f787a5c61)

          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/mover/Mover.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFs.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/AbstractFileSystem.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestHarFileSystem.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/StoragePolicyAdmin.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockStoragePolicy.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFileSystem.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileContext.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/fs/Hdfs.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/BlockStoragePolicySpi.java
          • hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/BlockStoragePolicy.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ChRootedFs.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ViewFs.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk-Java8 #201 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/201/ ) HDFS-8345 . Storage policy APIs must be exposed via the FileSystem interface. (Arpit Agarwal) (arp: rev a2190bf15d25e01fb4b220ba6401ce2f787a5c61) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/mover/Mover.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFs.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/AbstractFileSystem.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestHarFileSystem.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/StoragePolicyAdmin.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockStoragePolicy.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFileSystem.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileContext.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/fs/Hdfs.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/BlockStoragePolicySpi.java hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/BlockStoragePolicy.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ChRootedFs.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ViewFs.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk #2130 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2130/)
          HDFS-8345. Storage policy APIs must be exposed via the FileSystem interface. (Arpit Agarwal) (arp: rev a2190bf15d25e01fb4b220ba6401ce2f787a5c61)

          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestHarFileSystem.java
          • hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/BlockStoragePolicy.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFs.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ChRootedFs.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ViewFs.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/mover/Mover.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/fs/Hdfs.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/StoragePolicyAdmin.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/BlockStoragePolicySpi.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockStoragePolicy.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFileSystem.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/AbstractFileSystem.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileContext.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2130 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2130/ ) HDFS-8345 . Storage policy APIs must be exposed via the FileSystem interface. (Arpit Agarwal) (arp: rev a2190bf15d25e01fb4b220ba6401ce2f787a5c61) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestHarFileSystem.java hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/BlockStoragePolicy.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFs.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ChRootedFs.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ViewFs.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/mover/Mover.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/fs/Hdfs.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/StoragePolicyAdmin.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/BlockStoragePolicySpi.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockStoragePolicy.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFileSystem.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/AbstractFileSystem.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileContext.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #190 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/190/)
          HDFS-8345. Storage policy APIs must be exposed via the FileSystem interface. (Arpit Agarwal) (arp: rev a2190bf15d25e01fb4b220ba6401ce2f787a5c61)

          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/AbstractFileSystem.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ChRootedFs.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFileSystem.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/BlockStoragePolicySpi.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockStoragePolicy.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestHarFileSystem.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/BlockStoragePolicy.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFs.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/fs/Hdfs.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileContext.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/mover/Mover.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/StoragePolicyAdmin.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ViewFs.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #190 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/190/ ) HDFS-8345 . Storage policy APIs must be exposed via the FileSystem interface. (Arpit Agarwal) (arp: rev a2190bf15d25e01fb4b220ba6401ce2f787a5c61) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/AbstractFileSystem.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ChRootedFs.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFileSystem.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/BlockStoragePolicySpi.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockStoragePolicy.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestHarFileSystem.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/BlockStoragePolicy.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFs.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/fs/Hdfs.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileContext.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/mover/Mover.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/StoragePolicyAdmin.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ViewFs.java
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Mapreduce-trunk-Java8 #200 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/200/)
          HDFS-8345. Storage policy APIs must be exposed via the FileSystem interface. (Arpit Agarwal) (arp: rev a2190bf15d25e01fb4b220ba6401ce2f787a5c61)

          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ChRootedFs.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFileSystem.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/StoragePolicyAdmin.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFs.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/BlockStoragePolicySpi.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileContext.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ViewFs.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/mover/Mover.java
          • hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/BlockStoragePolicy.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockStoragePolicy.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/fs/Hdfs.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestHarFileSystem.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/AbstractFileSystem.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Mapreduce-trunk-Java8 #200 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/200/ ) HDFS-8345 . Storage policy APIs must be exposed via the FileSystem interface. (Arpit Agarwal) (arp: rev a2190bf15d25e01fb4b220ba6401ce2f787a5c61) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ChRootedFs.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFileSystem.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/StoragePolicyAdmin.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFs.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/BlockStoragePolicySpi.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileContext.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ViewFs.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/mover/Mover.java hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/BlockStoragePolicy.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockStoragePolicy.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/fs/Hdfs.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestHarFileSystem.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/AbstractFileSystem.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk #2148 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2148/)
          HDFS-8345. Storage policy APIs must be exposed via the FileSystem interface. (Arpit Agarwal) (arp: rev a2190bf15d25e01fb4b220ba6401ce2f787a5c61)

          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/mover/Mover.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/BlockStoragePolicySpi.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/fs/Hdfs.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/AbstractFileSystem.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestHarFileSystem.java
          • hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/BlockStoragePolicy.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFileSystem.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ViewFs.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileContext.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockStoragePolicy.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFs.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ChRootedFs.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/StoragePolicyAdmin.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2148 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2148/ ) HDFS-8345 . Storage policy APIs must be exposed via the FileSystem interface. (Arpit Agarwal) (arp: rev a2190bf15d25e01fb4b220ba6401ce2f787a5c61) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/mover/Mover.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/BlockStoragePolicySpi.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/fs/Hdfs.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/AbstractFileSystem.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestHarFileSystem.java hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/BlockStoragePolicy.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFileSystem.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ViewFs.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileContext.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockStoragePolicy.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFs.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ChRootedFs.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/StoragePolicyAdmin.java

            People

            • Assignee:
              arpitagarwal Arpit Agarwal
              Reporter:
              arpitagarwal Arpit Agarwal
            • Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development