Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-13828

Implement getFileChecksum(path, length) for ViewFileSystem

    Details

    • Type: Task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.0.0-alpha1
    • Fix Version/s: 3.0.0-alpha2
    • Component/s: viewfs
    • Labels:
      None

      Description

      ViewFileSystem inherits the default implementation of getFileChecksum(final Path f, final long length) from FileSystem which returns null. ViewFileSystem must override this to resolve the target filesystem and file path from configured mount points and invoke the right checksum method on the target filesystem.

      1. HADOOP-13828.01.patch
        4 kB
        Manoj Govindassamy

        Activity

        Hide
        manojg Manoj Govindassamy added a comment -

        Attaching v01 patch to take care of the following

        • Implemented getFileChecksum(path, length) for ViewFileSystem
        • Test to verify file checksum via ViewFileSystem is same as the target FiileSystem
        • Test needs the ChRootedFileSystem also implement the similar FileSystem's getFileChecksum contract and it is folded in along with this change. Without ChRootedFileSystem change, the test will fail.

        Andrew Wang, kindly review the patch.

        Show
        manojg Manoj Govindassamy added a comment - Attaching v01 patch to take care of the following Implemented getFileChecksum(path, length) for ViewFileSystem Test to verify file checksum via ViewFileSystem is same as the target FiileSystem Test needs the ChRootedFileSystem also implement the similar FileSystem's getFileChecksum contract and it is folded in along with this change. Without ChRootedFileSystem change, the test will fail. Andrew Wang , kindly review the patch.
        Hide
        andrew.wang Andrew Wang added a comment -

        LGTM +1 pending Jenkins, thanks Manoj!

        Show
        andrew.wang Andrew Wang added a comment - LGTM +1 pending Jenkins, thanks Manoj!
        Hide
        hadoopqa Hadoop QA added a comment -
        +1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 11s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        0 mvndep 0m 17s Maven dependency ordering for branch
        +1 mvninstall 8m 9s trunk passed
        +1 compile 9m 59s trunk passed
        +1 checkstyle 2m 2s trunk passed
        +1 mvnsite 2m 13s trunk passed
        +1 mvneclipse 0m 38s trunk passed
        +1 findbugs 3m 25s trunk passed
        +1 javadoc 1m 36s trunk passed
        0 mvndep 0m 18s Maven dependency ordering for patch
        +1 mvninstall 1m 29s the patch passed
        +1 compile 9m 35s the patch passed
        +1 javac 9m 35s the patch passed
        +1 checkstyle 1m 37s the patch passed
        +1 mvnsite 2m 5s the patch passed
        +1 mvneclipse 0m 44s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 3m 38s the patch passed
        +1 javadoc 1m 44s the patch passed
        +1 unit 8m 5s hadoop-common in the patch passed.
        +1 unit 67m 39s hadoop-hdfs in the patch passed.
        +1 asflicense 0m 45s The patch does not generate ASF License warnings.
        150m 3s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:a9ad5d6
        JIRA Issue HADOOP-13828
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12840344/HADOOP-13828.01.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 2894d1cb427d 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 0de0c32
        Default Java 1.8.0_111
        findbugs v3.0.0
        Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/11125/testReport/
        modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: .
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11125/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 11s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. 0 mvndep 0m 17s Maven dependency ordering for branch +1 mvninstall 8m 9s trunk passed +1 compile 9m 59s trunk passed +1 checkstyle 2m 2s trunk passed +1 mvnsite 2m 13s trunk passed +1 mvneclipse 0m 38s trunk passed +1 findbugs 3m 25s trunk passed +1 javadoc 1m 36s trunk passed 0 mvndep 0m 18s Maven dependency ordering for patch +1 mvninstall 1m 29s the patch passed +1 compile 9m 35s the patch passed +1 javac 9m 35s the patch passed +1 checkstyle 1m 37s the patch passed +1 mvnsite 2m 5s the patch passed +1 mvneclipse 0m 44s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 3m 38s the patch passed +1 javadoc 1m 44s the patch passed +1 unit 8m 5s hadoop-common in the patch passed. +1 unit 67m 39s hadoop-hdfs in the patch passed. +1 asflicense 0m 45s The patch does not generate ASF License warnings. 150m 3s Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HADOOP-13828 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12840344/HADOOP-13828.01.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 2894d1cb427d 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 0de0c32 Default Java 1.8.0_111 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/11125/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11125/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        andrew.wang Andrew Wang added a comment -

        Thanks again for the patch Manoj! I had to do a small rebase for the test since the df patch was committed after the Jenkins run. Committed to trunk.

        Show
        andrew.wang Andrew Wang added a comment - Thanks again for the patch Manoj! I had to do a small rebase for the test since the df patch was committed after the Jenkins run. Committed to trunk.
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10899 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10899/)
        HADOOP-13828. Implement getFileChecksum(path, length) for (wang: rev a2b1ff0257bde26d1f64454e97bc1225294a30b9)

        • (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ViewFileSystem.java
        • (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ChRootedFileSystem.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/fs/viewfs/TestViewFileSystemHdfs.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10899 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10899/ ) HADOOP-13828 . Implement getFileChecksum(path, length) for (wang: rev a2b1ff0257bde26d1f64454e97bc1225294a30b9) (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ViewFileSystem.java (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ChRootedFileSystem.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/fs/viewfs/TestViewFileSystemHdfs.java

          People

          • Assignee:
            manojg Manoj Govindassamy
            Reporter:
            manojg Manoj Govindassamy
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development