Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-12077

Provide a multi-URI replication Inode for ViewFs

    Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.0.0-beta1
    • Component/s: fs
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      This JIRA is to provide simple "replication" capabilities for applications that maintain logically equivalent paths in multiple locations for caching or failover (e.g., S3 and HDFS). We noticed a simple common HDFS usage pattern in our applications. They host their data on some logical cluster C. There are corresponding HDFS clusters in multiple datacenters. When the application runs in DC1, it prefers to read from C in DC1, and the applications prefers to failover to C in DC2 if the application is migrated to DC2 or when C in DC1 is unavailable. New application data versions are created periodically/relatively infrequently.

      In order to address many common scenarios in a general fashion, and to avoid unnecessary code duplication, we implement this functionality in ViewFs (our default FileSystem spanning all clusters in all datacenters) in a project code-named Nfly (N as in N datacenters). Currently each ViewFs Inode points to a single URI via ChRootedFileSystem. Consequently, we introduce a new type of links that points to a list of URIs that are each going to be wrapped in ChRootedFileSystem. A typical usage: /nfly/C/user->/DC1/C/user,/DC2/C/user,... This collection of ChRootedFileSystem instances is fronted by the Nfly filesystem object that is actually used for the mount point/Inode. Nfly filesystems backs a single logical path /nfly/C/user/<user>/path by multiple physical paths.

      Nfly filesystem supports setting minReplication. As long as the number of URIs on which an update has succeeded is greater than or equal to minReplication exceptions are only logged but not thrown. Each update operation is currently executed serially (client-bandwidth driven parallelism will be added later).

      A file create/write:

      1. Creates a temporary invisible _nfly_tmp_file in the intended chrooted filesystem.
      2. Returns a FSDataOutputStream that wraps output streams returned by 1
      3. All writes are forwarded to each output stream.
      4. On close of stream created by 2, all n streams are closed, and the files are renamed from _nfly_tmp_file to file. All files receive the same mtime corresponding to the client system time as of beginning of this step.
      5. If at least minReplication destinations has gone through steps 1-4 without failures the transaction is considered logically committed, otherwise a best-effort attempt of cleaning up the temporary files is attempted.

      As for reads, we support a notion of locality similar to HDFS /DC/rack/node. We sort Inode URIs using NetworkTopology by their authorities. These are typically host names in simple HDFS URIs. If the authority is missing as is the case with the local file:/// the local host name is assumed InetAddress.getLocalHost(). This makes sure that the local file system is always the closest one to the reader in this approach. For our Hadoop 2 hdfs URIs that are based on nameservice ids instead of hostnames it is very easy to adjust the topology script since our nameservice ids already contain the datacenter. As for rack and node we can simply output any string such as /DC/rack-nsid/node-nsid, since we only care about datacenter-locality for such filesystem clients.

      There are 2 policies/additions to the read call path that makes it more expensive, but improve user experience:

      • readMostRecent - when this policy is enabled, Nfly first checks mtime for the path under all URIs, sorts them from most recent to least recent. Nfly then sorts the set of most recent URIs topologically in the same manner as described above.
      • repairOnRead - when readMostRecent is enabled Nfly already has to RPC all underlying destinations. With repairOnRead, Nfly filesystem would additionally attempt to refresh destinations with the path missing or a stale version of the path using the nearest available most recent destination.
      1. HADOOP-12077.010.patch
        58 kB
        Gera Shegalov
      2. HADOOP-12077.009.patch
        56 kB
        Chris Douglas
      3. HADOOP-12077.008.patch
        56 kB
        Chris Douglas
      4. HADOOP-12077.007.patch
        55 kB
        Chris Douglas
      5. HADOOP-12077.006.patch
        55 kB
        Chris Douglas
      6. HADOOP-12077.005.patch
        53 kB
        Zhe Zhang
      7. HADOOP-12077.004.patch
        53 kB
        Gera Shegalov
      8. HADOOP-12077.003.patch
        52 kB
        Gera Shegalov
      9. HADOOP-12077.002.patch
        52 kB
        Gera Shegalov
      10. HADOOP-12077.001.patch
        51 kB
        Gera Shegalov

        Issue Links

          Activity

          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 19m 27s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 3 new or modified test files.
          +1 javac 7m 34s There were no new javac warning messages.
          +1 javadoc 9m 40s There were no new javadoc warning messages.
          +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 2m 14s The applied patch generated 37 new checkstyle issues (total was 179, now 215).
          -1 whitespace 0m 2s The patch has 4 line(s) that end in whitespace. Use git apply --whitespace=fix.
          +1 install 1m 36s mvn install still works.
          +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
          -1 findbugs 5m 7s The patch appears to introduce 3 new Findbugs (version 3.0.0) warnings.
          +1 common tests 22m 51s Tests passed in hadoop-common.
          -1 hdfs tests 161m 22s Tests failed in hadoop-hdfs.
              231m 23s  



          Reason Tests
          FindBugs module:hadoop-common
          Failed unit tests hadoop.hdfs.server.namenode.TestClusterId



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12738534/HADOOP-12077.001.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 8d0ef31
          checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/6941/artifact/patchprocess/diffcheckstylehadoop-common.txt
          whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/6941/artifact/patchprocess/whitespace.txt
          Findbugs warnings https://builds.apache.org/job/PreCommit-HADOOP-Build/6941/artifact/patchprocess/newPatchFindbugsWarningshadoop-common.html
          hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/6941/artifact/patchprocess/testrun_hadoop-common.txt
          hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HADOOP-Build/6941/artifact/patchprocess/testrun_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/6941/testReport/
          Java 1.7.0_55
          uname Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/6941/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 19m 27s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 3 new or modified test files. +1 javac 7m 34s There were no new javac warning messages. +1 javadoc 9m 40s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 2m 14s The applied patch generated 37 new checkstyle issues (total was 179, now 215). -1 whitespace 0m 2s The patch has 4 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 install 1m 36s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. -1 findbugs 5m 7s The patch appears to introduce 3 new Findbugs (version 3.0.0) warnings. +1 common tests 22m 51s Tests passed in hadoop-common. -1 hdfs tests 161m 22s Tests failed in hadoop-hdfs.     231m 23s   Reason Tests FindBugs module:hadoop-common Failed unit tests hadoop.hdfs.server.namenode.TestClusterId Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12738534/HADOOP-12077.001.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 8d0ef31 checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/6941/artifact/patchprocess/diffcheckstylehadoop-common.txt whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/6941/artifact/patchprocess/whitespace.txt Findbugs warnings https://builds.apache.org/job/PreCommit-HADOOP-Build/6941/artifact/patchprocess/newPatchFindbugsWarningshadoop-common.html hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/6941/artifact/patchprocess/testrun_hadoop-common.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HADOOP-Build/6941/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/6941/testReport/ Java 1.7.0_55 uname Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/6941/console This message was automatically generated.
          Hide
          jira.shegalov Gera Shegalov added a comment -

          Addressed a bulk of check style warnings. Test failures are unrelated and don't reproduce locally.

          Show
          jira.shegalov Gera Shegalov added a comment - Addressed a bulk of check style warnings. Test failures are unrelated and don't reproduce locally.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 19m 52s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 3 new or modified test files.
          +1 javac 7m 38s There were no new javac warning messages.
          +1 javadoc 9m 43s There were no new javadoc warning messages.
          +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings.
          +1 checkstyle 2m 46s There were no new checkstyle issues.
          -1 whitespace 0m 2s The patch has 2 line(s) that end in whitespace. Use git apply --whitespace=fix.
          +1 install 1m 41s mvn install still works.
          +1 eclipse:eclipse 0m 36s The patch built with eclipse:eclipse.
          -1 findbugs 5m 13s The patch appears to introduce 3 new Findbugs (version 3.0.0) warnings.
          -1 common tests 23m 43s Tests failed in hadoop-common.
          +1 hdfs tests 162m 22s Tests passed in hadoop-hdfs.
              234m 10s  



          Reason Tests
          FindBugs module:hadoop-common
          Failed unit tests hadoop.fs.shell.TestCount



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12739737/HADOOP-12077.002.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 175e6d1
          whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/6969/artifact/patchprocess/whitespace.txt
          Findbugs warnings https://builds.apache.org/job/PreCommit-HADOOP-Build/6969/artifact/patchprocess/newPatchFindbugsWarningshadoop-common.html
          hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/6969/artifact/patchprocess/testrun_hadoop-common.txt
          hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HADOOP-Build/6969/artifact/patchprocess/testrun_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/6969/testReport/
          Java 1.7.0_55
          uname Linux asf901.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/6969/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 19m 52s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 3 new or modified test files. +1 javac 7m 38s There were no new javac warning messages. +1 javadoc 9m 43s There were no new javadoc warning messages. +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 2m 46s There were no new checkstyle issues. -1 whitespace 0m 2s The patch has 2 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 install 1m 41s mvn install still works. +1 eclipse:eclipse 0m 36s The patch built with eclipse:eclipse. -1 findbugs 5m 13s The patch appears to introduce 3 new Findbugs (version 3.0.0) warnings. -1 common tests 23m 43s Tests failed in hadoop-common. +1 hdfs tests 162m 22s Tests passed in hadoop-hdfs.     234m 10s   Reason Tests FindBugs module:hadoop-common Failed unit tests hadoop.fs.shell.TestCount Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12739737/HADOOP-12077.002.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 175e6d1 whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/6969/artifact/patchprocess/whitespace.txt Findbugs warnings https://builds.apache.org/job/PreCommit-HADOOP-Build/6969/artifact/patchprocess/newPatchFindbugsWarningshadoop-common.html hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/6969/artifact/patchprocess/testrun_hadoop-common.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HADOOP-Build/6969/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/6969/testReport/ Java 1.7.0_55 uname Linux asf901.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/6969/console This message was automatically generated.
          Hide
          jira.shegalov Gera Shegalov added a comment -

          003 to further get rid of checkstyle issues. HADOOP-12095 covers the test failure.

          Show
          jira.shegalov Gera Shegalov added a comment - 003 to further get rid of checkstyle issues. HADOOP-12095 covers the test failure.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          -1 pre-patch 19m 10s Findbugs (version 3.0.0) appears to be broken on trunk.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 3 new or modified test files.
          +1 javac 7m 49s There were no new javac warning messages.
          +1 javadoc 9m 57s There were no new javadoc warning messages.
          +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 2m 14s The applied patch generated 176 new checkstyle issues (total was 0, now 176).
          -1 whitespace 0m 2s The patch has 2 line(s) that end in whitespace. Use git apply --whitespace=fix.
          +1 install 1m 38s mvn install still works.
          +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
          -1 findbugs 5m 13s The patch appears to introduce 3 new Findbugs (version 3.0.0) warnings.
          +1 common tests 22m 2s Tests passed in hadoop-common.
          -1 hdfs tests 161m 48s Tests failed in hadoop-hdfs.
              231m 23s  



          Reason Tests
          FindBugs module:hadoop-common
          Failed unit tests hadoop.hdfs.qjournal.TestSecureNNWithQJM
            hadoop.hdfs.TestDFSStartupVersions
            hadoop.fs.viewfs.TestViewFileSystemHdfs



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12742062/HADOOP-12077.003.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 8ef07f7
          checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/7051/artifact/patchprocess/diffcheckstylehadoop-common.txt
          whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/7051/artifact/patchprocess/whitespace.txt
          Findbugs warnings https://builds.apache.org/job/PreCommit-HADOOP-Build/7051/artifact/patchprocess/newPatchFindbugsWarningshadoop-common.html
          hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7051/artifact/patchprocess/testrun_hadoop-common.txt
          hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7051/artifact/patchprocess/testrun_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/7051/testReport/
          Java 1.7.0_55
          uname Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7051/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 pre-patch 19m 10s Findbugs (version 3.0.0) appears to be broken on trunk. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 3 new or modified test files. +1 javac 7m 49s There were no new javac warning messages. +1 javadoc 9m 57s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 2m 14s The applied patch generated 176 new checkstyle issues (total was 0, now 176). -1 whitespace 0m 2s The patch has 2 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 install 1m 38s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. -1 findbugs 5m 13s The patch appears to introduce 3 new Findbugs (version 3.0.0) warnings. +1 common tests 22m 2s Tests passed in hadoop-common. -1 hdfs tests 161m 48s Tests failed in hadoop-hdfs.     231m 23s   Reason Tests FindBugs module:hadoop-common Failed unit tests hadoop.hdfs.qjournal.TestSecureNNWithQJM   hadoop.hdfs.TestDFSStartupVersions   hadoop.fs.viewfs.TestViewFileSystemHdfs Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12742062/HADOOP-12077.003.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 8ef07f7 checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/7051/artifact/patchprocess/diffcheckstylehadoop-common.txt whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/7051/artifact/patchprocess/whitespace.txt Findbugs warnings https://builds.apache.org/job/PreCommit-HADOOP-Build/7051/artifact/patchprocess/newPatchFindbugsWarningshadoop-common.html hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7051/artifact/patchprocess/testrun_hadoop-common.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7051/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/7051/testReport/ Java 1.7.0_55 uname Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7051/console This message was automatically generated.
          Hide
          jira.shegalov Gera Shegalov added a comment -

          Early validation of minReplication in the constructor.

          Show
          jira.shegalov Gera Shegalov added a comment - Early validation of minReplication in the constructor.
          Hide
          jira.shegalov Gera Shegalov added a comment -

          Jenkins never ran after 004. Cancelling and resubmitting.

          Show
          jira.shegalov Gera Shegalov added a comment - Jenkins never ran after 004. Cancelling and resubmitting.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          -1 pre-patch 17m 1s Findbugs (version ) appears to be broken on trunk.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 3 new or modified test files.
          +1 javac 7m 40s There were no new javac warning messages.
          +1 javadoc 9m 37s There were no new javadoc warning messages.
          +1 release audit 0m 21s The applied patch does not increase the total number of release audit warnings.
          +1 checkstyle 1m 50s There were no new checkstyle issues.
          +1 whitespace 0m 2s The patch has no lines that end in whitespace.
          +1 install 1m 31s mvn install still works.
          +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse.
          -1 findbugs 4m 21s The patch appears to introduce 3 new Findbugs (version 3.0.0) warnings.
          +1 common tests 22m 27s Tests passed in hadoop-common.
          +1 hdfs tests 158m 42s Tests passed in hadoop-hdfs.
              224m 8s  



          Reason Tests
          FindBugs module:hadoop-common



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12744920/HADOOP-12077.004.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / d7319de
          Findbugs warnings https://builds.apache.org/job/PreCommit-HADOOP-Build/7252/artifact/patchprocess/newPatchFindbugsWarningshadoop-common.html
          hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7252/artifact/patchprocess/testrun_hadoop-common.txt
          hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7252/artifact/patchprocess/testrun_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/7252/testReport/
          Java 1.7.0_55
          uname Linux asf901.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7252/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 pre-patch 17m 1s Findbugs (version ) appears to be broken on trunk. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 3 new or modified test files. +1 javac 7m 40s There were no new javac warning messages. +1 javadoc 9m 37s There were no new javadoc warning messages. +1 release audit 0m 21s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 1m 50s There were no new checkstyle issues. +1 whitespace 0m 2s The patch has no lines that end in whitespace. +1 install 1m 31s mvn install still works. +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse. -1 findbugs 4m 21s The patch appears to introduce 3 new Findbugs (version 3.0.0) warnings. +1 common tests 22m 27s Tests passed in hadoop-common. +1 hdfs tests 158m 42s Tests passed in hadoop-hdfs.     224m 8s   Reason Tests FindBugs module:hadoop-common Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12744920/HADOOP-12077.004.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / d7319de Findbugs warnings https://builds.apache.org/job/PreCommit-HADOOP-Build/7252/artifact/patchprocess/newPatchFindbugsWarningshadoop-common.html hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7252/artifact/patchprocess/testrun_hadoop-common.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7252/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/7252/testReport/ Java 1.7.0_55 uname Linux asf901.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7252/console This message was automatically generated.
          Hide
          jira.shegalov Gera Shegalov added a comment -

          Since FindBugs -1 report does not actually report anything wrong, I consider 004 reviewable.

          Show
          jira.shegalov Gera Shegalov added a comment - Since FindBugs -1 report does not actually report anything wrong, I consider 004 reviewable.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 22m 11s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 3 new or modified test files.
          -1 javac 0m 39s The patch appears to cause the build to fail.



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12744920/HADOOP-12077.004.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / fdf02d1
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7762/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 22m 11s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 3 new or modified test files. -1 javac 0m 39s The patch appears to cause the build to fail. Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12744920/HADOOP-12077.004.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / fdf02d1 Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7762/console This message was automatically generated.
          Hide
          zhz Zhe Zhang added a comment -

          Thanks Gera for the work. Very interesting feature. The overall structure looks good. I haven't finished reviewing the full patch. Uploading a slightly rebased patch based on current trunk.

          Show
          zhz Zhe Zhang added a comment - Thanks Gera for the work. Very interesting feature. The overall structure looks good. I haven't finished reviewing the full patch. Uploading a slightly rebased patch based on current trunk.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 14s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
          0 mvndep 0m 15s Maven dependency ordering for branch
          +1 mvninstall 6m 56s trunk passed
          +1 compile 6m 16s trunk passed with JDK v1.8.0_77
          +1 compile 7m 17s trunk passed with JDK v1.7.0_95
          +1 checkstyle 1m 8s trunk passed
          +1 mvnsite 1m 48s trunk passed
          +1 mvneclipse 0m 27s trunk passed
          +1 findbugs 3m 33s trunk passed
          +1 javadoc 2m 0s trunk passed with JDK v1.8.0_77
          +1 javadoc 2m 49s trunk passed with JDK v1.7.0_95
          0 mvndep 0m 13s Maven dependency ordering for patch
          +1 mvninstall 1m 32s the patch passed
          +1 compile 6m 42s the patch passed with JDK v1.8.0_77
          +1 javac 6m 42s the patch passed
          +1 compile 6m 58s the patch passed with JDK v1.7.0_95
          +1 javac 6m 58s the patch passed
          -1 checkstyle 1m 5s root: patch generated 2 new + 169 unchanged - 4 fixed = 171 total (was 173)
          +1 mvnsite 1m 47s the patch passed
          +1 mvneclipse 0m 27s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          -1 findbugs 1m 53s hadoop-common-project/hadoop-common generated 4 new + 0 unchanged - 0 fixed = 4 total (was 0)
          +1 javadoc 2m 0s the patch passed with JDK v1.8.0_77
          +1 javadoc 2m 48s the patch passed with JDK v1.7.0_95
          -1 unit 19m 51s hadoop-common in the patch failed with JDK v1.8.0_77.
          -1 unit 73m 0s hadoop-hdfs in the patch failed with JDK v1.8.0_77.
          +1 unit 9m 44s hadoop-common in the patch passed with JDK v1.7.0_95.
          -1 unit 78m 1s hadoop-hdfs in the patch failed with JDK v1.7.0_95.
          +1 asflicense 0m 42s Patch does not generate ASF License warnings.
          243m 2s



          Reason Tests
          FindBugs module:hadoop-common-project/hadoop-common
            Boxing/unboxing to parse a primitive org.apache.hadoop.fs.viewfs.NflyFSystem.createFileSystem(URI[], Configuration, String) At NflyFSystem.java:org.apache.hadoop.fs.viewfs.NflyFSystem.createFileSystem(URI[], Configuration, String) At NflyFSystem.java:[line 933]
            org.apache.hadoop.fs.viewfs.NflyFSystem$MRNflyNode doesn't override org.apache.hadoop.net.NodeBase.equals(Object) At NflyFSystem.java:At NflyFSystem.java:[line 1]
            org.apache.hadoop.fs.viewfs.NflyFSystem$NflyNode doesn't override org.apache.hadoop.net.NodeBase.equals(Object) At NflyFSystem.java:At NflyFSystem.java:[line 1]
            org.apache.hadoop.fs.viewfs.NflyFSystem$NflyStatus overrides equals in org.apache.hadoop.fs.FileStatus and may not be symmetric At NflyFSystem.java:and may not be symmetric At NflyFSystem.java:[lines 523-526]
          JDK v1.8.0_77 Failed junit tests hadoop.hdfs.TestReadStripedFileWithMissingBlocks
            hadoop.hdfs.shortcircuit.TestShortCircuitLocalRead
            hadoop.hdfs.TestFileAppend
            hadoop.hdfs.TestDFSStripedOutputStreamWithFailure
          JDK v1.8.0_77 Timed out junit tests org.apache.hadoop.http.TestHttpServerLifecycle
            org.apache.hadoop.hdfs.TestWriteReadStripedFile
            org.apache.hadoop.hdfs.TestReadStripedFileWithDecoding
          JDK v1.7.0_95 Failed junit tests hadoop.hdfs.TestAclsEndToEnd
            hadoop.hdfs.server.datanode.TestDataNodeUUID
            hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints
            hadoop.hdfs.TestReadStripedFileWithMissingBlocks
            hadoop.hdfs.shortcircuit.TestShortCircuitLocalRead
            hadoop.hdfs.server.datanode.TestDirectoryScanner
            hadoop.hdfs.TestCrcCorruption
            hadoop.hdfs.TestEncryptionZones
            hadoop.hdfs.TestHFlush
            hadoop.hdfs.server.namenode.TestLargeDirectoryDelete
            hadoop.hdfs.TestDFSStripedOutputStreamWithFailure
          JDK v1.7.0_95 Timed out junit tests org.apache.hadoop.hdfs.TestWriteReadStripedFile
            org.apache.hadoop.hdfs.TestReadStripedFileWithDecoding



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:fbe3e86
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12798359/HADOOP-12077.005.patch
          JIRA Issue HADOOP-12077
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 814d7da5e2e0 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 6ef4287
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/9074/artifact/patchprocess/diff-checkstyle-root.txt
          findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/9074/artifact/patchprocess/new-findbugs-hadoop-common-project_hadoop-common.html
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9074/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_77.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9074/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_77.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9074/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt
          unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/9074/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_77.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/9074/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_77.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/9074/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9074/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9074/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 14s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files. 0 mvndep 0m 15s Maven dependency ordering for branch +1 mvninstall 6m 56s trunk passed +1 compile 6m 16s trunk passed with JDK v1.8.0_77 +1 compile 7m 17s trunk passed with JDK v1.7.0_95 +1 checkstyle 1m 8s trunk passed +1 mvnsite 1m 48s trunk passed +1 mvneclipse 0m 27s trunk passed +1 findbugs 3m 33s trunk passed +1 javadoc 2m 0s trunk passed with JDK v1.8.0_77 +1 javadoc 2m 49s trunk passed with JDK v1.7.0_95 0 mvndep 0m 13s Maven dependency ordering for patch +1 mvninstall 1m 32s the patch passed +1 compile 6m 42s the patch passed with JDK v1.8.0_77 +1 javac 6m 42s the patch passed +1 compile 6m 58s the patch passed with JDK v1.7.0_95 +1 javac 6m 58s the patch passed -1 checkstyle 1m 5s root: patch generated 2 new + 169 unchanged - 4 fixed = 171 total (was 173) +1 mvnsite 1m 47s the patch passed +1 mvneclipse 0m 27s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. -1 findbugs 1m 53s hadoop-common-project/hadoop-common generated 4 new + 0 unchanged - 0 fixed = 4 total (was 0) +1 javadoc 2m 0s the patch passed with JDK v1.8.0_77 +1 javadoc 2m 48s the patch passed with JDK v1.7.0_95 -1 unit 19m 51s hadoop-common in the patch failed with JDK v1.8.0_77. -1 unit 73m 0s hadoop-hdfs in the patch failed with JDK v1.8.0_77. +1 unit 9m 44s hadoop-common in the patch passed with JDK v1.7.0_95. -1 unit 78m 1s hadoop-hdfs in the patch failed with JDK v1.7.0_95. +1 asflicense 0m 42s Patch does not generate ASF License warnings. 243m 2s Reason Tests FindBugs module:hadoop-common-project/hadoop-common   Boxing/unboxing to parse a primitive org.apache.hadoop.fs.viewfs.NflyFSystem.createFileSystem(URI[], Configuration, String) At NflyFSystem.java:org.apache.hadoop.fs.viewfs.NflyFSystem.createFileSystem(URI[], Configuration, String) At NflyFSystem.java: [line 933]   org.apache.hadoop.fs.viewfs.NflyFSystem$MRNflyNode doesn't override org.apache.hadoop.net.NodeBase.equals(Object) At NflyFSystem.java:At NflyFSystem.java: [line 1]   org.apache.hadoop.fs.viewfs.NflyFSystem$NflyNode doesn't override org.apache.hadoop.net.NodeBase.equals(Object) At NflyFSystem.java:At NflyFSystem.java: [line 1]   org.apache.hadoop.fs.viewfs.NflyFSystem$NflyStatus overrides equals in org.apache.hadoop.fs.FileStatus and may not be symmetric At NflyFSystem.java:and may not be symmetric At NflyFSystem.java: [lines 523-526] JDK v1.8.0_77 Failed junit tests hadoop.hdfs.TestReadStripedFileWithMissingBlocks   hadoop.hdfs.shortcircuit.TestShortCircuitLocalRead   hadoop.hdfs.TestFileAppend   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure JDK v1.8.0_77 Timed out junit tests org.apache.hadoop.http.TestHttpServerLifecycle   org.apache.hadoop.hdfs.TestWriteReadStripedFile   org.apache.hadoop.hdfs.TestReadStripedFileWithDecoding JDK v1.7.0_95 Failed junit tests hadoop.hdfs.TestAclsEndToEnd   hadoop.hdfs.server.datanode.TestDataNodeUUID   hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints   hadoop.hdfs.TestReadStripedFileWithMissingBlocks   hadoop.hdfs.shortcircuit.TestShortCircuitLocalRead   hadoop.hdfs.server.datanode.TestDirectoryScanner   hadoop.hdfs.TestCrcCorruption   hadoop.hdfs.TestEncryptionZones   hadoop.hdfs.TestHFlush   hadoop.hdfs.server.namenode.TestLargeDirectoryDelete   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure JDK v1.7.0_95 Timed out junit tests org.apache.hadoop.hdfs.TestWriteReadStripedFile   org.apache.hadoop.hdfs.TestReadStripedFileWithDecoding Subsystem Report/Notes Docker Image:yetus/hadoop:fbe3e86 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12798359/HADOOP-12077.005.patch JIRA Issue HADOOP-12077 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 814d7da5e2e0 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 6ef4287 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/9074/artifact/patchprocess/diff-checkstyle-root.txt findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/9074/artifact/patchprocess/new-findbugs-hadoop-common-project_hadoop-common.html unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9074/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_77.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9074/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_77.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9074/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/9074/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_77.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/9074/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_77.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/9074/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9074/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9074/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 26s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
          0 mvndep 0m 33s Maven dependency ordering for branch
          +1 mvninstall 6m 49s trunk passed
          +1 compile 7m 0s trunk passed
          +1 checkstyle 1m 23s trunk passed
          +1 mvnsite 1m 51s trunk passed
          +1 mvneclipse 0m 27s trunk passed
          +1 findbugs 3m 17s trunk passed
          +1 javadoc 1m 46s trunk passed
          0 mvndep 0m 14s Maven dependency ordering for patch
          +1 mvninstall 1m 35s the patch passed
          +1 compile 6m 54s the patch passed
          -1 javac 6m 54s root generated 1 new + 708 unchanged - 0 fixed = 709 total (was 708)
          -0 checkstyle 1m 26s root: The patch generated 7 new + 180 unchanged - 8 fixed = 187 total (was 188)
          +1 mvnsite 1m 57s the patch passed
          +1 mvneclipse 0m 34s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          -1 findbugs 1m 40s hadoop-common-project/hadoop-common generated 4 new + 0 unchanged - 0 fixed = 4 total (was 0)
          +1 javadoc 2m 0s the patch passed
          +1 unit 7m 31s hadoop-common in the patch passed.
          -1 unit 58m 39s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 35s The patch does not generate ASF License warnings.
          131m 20s



          Reason Tests
          FindBugs module:hadoop-common-project/hadoop-common
            Boxing/unboxing to parse a primitive org.apache.hadoop.fs.viewfs.NflyFSystem.createFileSystem(URI[], Configuration, String) At NflyFSystem.java:org.apache.hadoop.fs.viewfs.NflyFSystem.createFileSystem(URI[], Configuration, String) At NflyFSystem.java:[line 933]
            org.apache.hadoop.fs.viewfs.NflyFSystem$MRNflyNode doesn't override org.apache.hadoop.net.NodeBase.equals(Object) At NflyFSystem.java:At NflyFSystem.java:[line 1]
            org.apache.hadoop.fs.viewfs.NflyFSystem$NflyNode doesn't override org.apache.hadoop.net.NodeBase.equals(Object) At NflyFSystem.java:At NflyFSystem.java:[line 1]
            org.apache.hadoop.fs.viewfs.NflyFSystem$NflyStatus overrides equals in org.apache.hadoop.fs.FileStatus and may not be symmetric At NflyFSystem.java:and may not be symmetric At NflyFSystem.java:[lines 523-526]
          Failed junit tests hadoop.hdfs.tools.offlineEditsViewer.TestOfflineEditsViewer



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:85209cc
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12798359/HADOOP-12077.005.patch
          JIRA Issue HADOOP-12077
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux fed46bb3cdce 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 846ada2
          Default Java 1.8.0_91
          findbugs v3.0.0
          javac https://builds.apache.org/job/PreCommit-HADOOP-Build/9908/artifact/patchprocess/diff-compile-javac-root.txt
          checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/9908/artifact/patchprocess/diff-checkstyle-root.txt
          findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/9908/artifact/patchprocess/new-findbugs-hadoop-common-project_hadoop-common.html
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9908/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9908/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9908/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 26s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files. 0 mvndep 0m 33s Maven dependency ordering for branch +1 mvninstall 6m 49s trunk passed +1 compile 7m 0s trunk passed +1 checkstyle 1m 23s trunk passed +1 mvnsite 1m 51s trunk passed +1 mvneclipse 0m 27s trunk passed +1 findbugs 3m 17s trunk passed +1 javadoc 1m 46s trunk passed 0 mvndep 0m 14s Maven dependency ordering for patch +1 mvninstall 1m 35s the patch passed +1 compile 6m 54s the patch passed -1 javac 6m 54s root generated 1 new + 708 unchanged - 0 fixed = 709 total (was 708) -0 checkstyle 1m 26s root: The patch generated 7 new + 180 unchanged - 8 fixed = 187 total (was 188) +1 mvnsite 1m 57s the patch passed +1 mvneclipse 0m 34s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. -1 findbugs 1m 40s hadoop-common-project/hadoop-common generated 4 new + 0 unchanged - 0 fixed = 4 total (was 0) +1 javadoc 2m 0s the patch passed +1 unit 7m 31s hadoop-common in the patch passed. -1 unit 58m 39s hadoop-hdfs in the patch failed. +1 asflicense 0m 35s The patch does not generate ASF License warnings. 131m 20s Reason Tests FindBugs module:hadoop-common-project/hadoop-common   Boxing/unboxing to parse a primitive org.apache.hadoop.fs.viewfs.NflyFSystem.createFileSystem(URI[], Configuration, String) At NflyFSystem.java:org.apache.hadoop.fs.viewfs.NflyFSystem.createFileSystem(URI[], Configuration, String) At NflyFSystem.java: [line 933]   org.apache.hadoop.fs.viewfs.NflyFSystem$MRNflyNode doesn't override org.apache.hadoop.net.NodeBase.equals(Object) At NflyFSystem.java:At NflyFSystem.java: [line 1]   org.apache.hadoop.fs.viewfs.NflyFSystem$NflyNode doesn't override org.apache.hadoop.net.NodeBase.equals(Object) At NflyFSystem.java:At NflyFSystem.java: [line 1]   org.apache.hadoop.fs.viewfs.NflyFSystem$NflyStatus overrides equals in org.apache.hadoop.fs.FileStatus and may not be symmetric At NflyFSystem.java:and may not be symmetric At NflyFSystem.java: [lines 523-526] Failed junit tests hadoop.hdfs.tools.offlineEditsViewer.TestOfflineEditsViewer Subsystem Report/Notes Docker Image:yetus/hadoop:85209cc JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12798359/HADOOP-12077.005.patch JIRA Issue HADOOP-12077 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux fed46bb3cdce 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 846ada2 Default Java 1.8.0_91 findbugs v3.0.0 javac https://builds.apache.org/job/PreCommit-HADOOP-Build/9908/artifact/patchprocess/diff-compile-javac-root.txt checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/9908/artifact/patchprocess/diff-checkstyle-root.txt findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/9908/artifact/patchprocess/new-findbugs-hadoop-common-project_hadoop-common.html unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9908/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9908/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9908/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 22s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
          0 mvndep 0m 12s Maven dependency ordering for branch
          +1 mvninstall 7m 5s trunk passed
          +1 compile 7m 23s trunk passed
          +1 checkstyle 1m 26s trunk passed
          +1 mvnsite 1m 59s trunk passed
          +1 mvneclipse 0m 26s trunk passed
          +1 findbugs 3m 22s trunk passed
          +1 javadoc 1m 45s trunk passed
          0 mvndep 0m 14s Maven dependency ordering for patch
          +1 mvninstall 1m 38s the patch passed
          +1 compile 7m 25s the patch passed
          -1 javac 7m 25s root generated 1 new + 709 unchanged - 0 fixed = 710 total (was 709)
          -0 checkstyle 1m 25s root: The patch generated 7 new + 180 unchanged - 8 fixed = 187 total (was 188)
          +1 mvnsite 1m 55s the patch passed
          +1 mvneclipse 0m 33s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          -1 findbugs 1m 37s hadoop-common-project/hadoop-common generated 4 new + 0 unchanged - 0 fixed = 4 total (was 0)
          +1 javadoc 2m 2s the patch passed
          -1 unit 9m 27s hadoop-common in the patch failed.
          -1 unit 62m 1s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 29s The patch does not generate ASF License warnings.
          136m 50s



          Reason Tests
          FindBugs module:hadoop-common-project/hadoop-common
            Boxing/unboxing to parse a primitive org.apache.hadoop.fs.viewfs.NflyFSystem.createFileSystem(URI[], Configuration, String) At NflyFSystem.java:org.apache.hadoop.fs.viewfs.NflyFSystem.createFileSystem(URI[], Configuration, String) At NflyFSystem.java:[line 933]
            org.apache.hadoop.fs.viewfs.NflyFSystem$MRNflyNode doesn't override org.apache.hadoop.net.NodeBase.equals(Object) At NflyFSystem.java:At NflyFSystem.java:[line 1]
            org.apache.hadoop.fs.viewfs.NflyFSystem$NflyNode doesn't override org.apache.hadoop.net.NodeBase.equals(Object) At NflyFSystem.java:At NflyFSystem.java:[line 1]
            org.apache.hadoop.fs.viewfs.NflyFSystem$NflyStatus overrides equals in org.apache.hadoop.fs.FileStatus and may not be symmetric At NflyFSystem.java:and may not be symmetric At NflyFSystem.java:[lines 523-526]
          Failed junit tests hadoop.ha.TestZKFailoverController
            hadoop.hdfs.server.datanode.TestDataNodeErasureCodingMetrics
            hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaRecovery
            hadoop.hdfs.tools.offlineEditsViewer.TestOfflineEditsViewer



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:85209cc
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12798359/HADOOP-12077.005.patch
          JIRA Issue HADOOP-12077
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 6995f3fb99d7 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 846ada2
          Default Java 1.8.0_91
          findbugs v3.0.0
          javac https://builds.apache.org/job/PreCommit-HADOOP-Build/9910/artifact/patchprocess/diff-compile-javac-root.txt
          checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/9910/artifact/patchprocess/diff-checkstyle-root.txt
          findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/9910/artifact/patchprocess/new-findbugs-hadoop-common-project_hadoop-common.html
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9910/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9910/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9910/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9910/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 22s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files. 0 mvndep 0m 12s Maven dependency ordering for branch +1 mvninstall 7m 5s trunk passed +1 compile 7m 23s trunk passed +1 checkstyle 1m 26s trunk passed +1 mvnsite 1m 59s trunk passed +1 mvneclipse 0m 26s trunk passed +1 findbugs 3m 22s trunk passed +1 javadoc 1m 45s trunk passed 0 mvndep 0m 14s Maven dependency ordering for patch +1 mvninstall 1m 38s the patch passed +1 compile 7m 25s the patch passed -1 javac 7m 25s root generated 1 new + 709 unchanged - 0 fixed = 710 total (was 709) -0 checkstyle 1m 25s root: The patch generated 7 new + 180 unchanged - 8 fixed = 187 total (was 188) +1 mvnsite 1m 55s the patch passed +1 mvneclipse 0m 33s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. -1 findbugs 1m 37s hadoop-common-project/hadoop-common generated 4 new + 0 unchanged - 0 fixed = 4 total (was 0) +1 javadoc 2m 2s the patch passed -1 unit 9m 27s hadoop-common in the patch failed. -1 unit 62m 1s hadoop-hdfs in the patch failed. +1 asflicense 0m 29s The patch does not generate ASF License warnings. 136m 50s Reason Tests FindBugs module:hadoop-common-project/hadoop-common   Boxing/unboxing to parse a primitive org.apache.hadoop.fs.viewfs.NflyFSystem.createFileSystem(URI[], Configuration, String) At NflyFSystem.java:org.apache.hadoop.fs.viewfs.NflyFSystem.createFileSystem(URI[], Configuration, String) At NflyFSystem.java: [line 933]   org.apache.hadoop.fs.viewfs.NflyFSystem$MRNflyNode doesn't override org.apache.hadoop.net.NodeBase.equals(Object) At NflyFSystem.java:At NflyFSystem.java: [line 1]   org.apache.hadoop.fs.viewfs.NflyFSystem$NflyNode doesn't override org.apache.hadoop.net.NodeBase.equals(Object) At NflyFSystem.java:At NflyFSystem.java: [line 1]   org.apache.hadoop.fs.viewfs.NflyFSystem$NflyStatus overrides equals in org.apache.hadoop.fs.FileStatus and may not be symmetric At NflyFSystem.java:and may not be symmetric At NflyFSystem.java: [lines 523-526] Failed junit tests hadoop.ha.TestZKFailoverController   hadoop.hdfs.server.datanode.TestDataNodeErasureCodingMetrics   hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaRecovery   hadoop.hdfs.tools.offlineEditsViewer.TestOfflineEditsViewer Subsystem Report/Notes Docker Image:yetus/hadoop:85209cc JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12798359/HADOOP-12077.005.patch JIRA Issue HADOOP-12077 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 6995f3fb99d7 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 846ada2 Default Java 1.8.0_91 findbugs v3.0.0 javac https://builds.apache.org/job/PreCommit-HADOOP-Build/9910/artifact/patchprocess/diff-compile-javac-root.txt checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/9910/artifact/patchprocess/diff-checkstyle-root.txt findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/9910/artifact/patchprocess/new-findbugs-hadoop-common-project_hadoop-common.html unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9910/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9910/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9910/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9910/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          -1 patch 0m 10s HADOOP-12077 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help.



          Subsystem Report/Notes
          JIRA Issue HADOOP-12077
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12798359/HADOOP-12077.005.patch
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/12235/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. -1 patch 0m 10s HADOOP-12077 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. Subsystem Report/Notes JIRA Issue HADOOP-12077 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12798359/HADOOP-12077.005.patch Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/12235/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          chris.douglas Chris Douglas added a comment -

          Rebased Gera Shegalov patch

          Show
          chris.douglas Chris Douglas added a comment - Rebased Gera Shegalov patch
          Hide
          jira.shegalov Gera Shegalov added a comment -

          Thank you Chris Douglas

          Show
          jira.shegalov Gera Shegalov added a comment - Thank you Chris Douglas
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 19s Docker mode activated.
                Prechecks
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
                trunk Compile Tests
          0 mvndep 1m 27s Maven dependency ordering for branch
          +1 mvninstall 15m 15s trunk passed
          +1 compile 15m 57s trunk passed
          +1 checkstyle 2m 5s trunk passed
          +1 mvnsite 2m 34s trunk passed
          -1 findbugs 1m 54s hadoop-hdfs-project/hadoop-hdfs in trunk has 9 extant Findbugs warnings.
          +1 javadoc 1m 42s trunk passed
                Patch Compile Tests
          0 mvndep 0m 17s Maven dependency ordering for patch
          +1 mvninstall 1m 40s the patch passed
          +1 compile 12m 5s the patch passed
          -1 javac 12m 5s root generated 5 new + 1418 unchanged - 0 fixed = 1423 total (was 1418)
          -0 checkstyle 2m 2s root: The patch generated 11 new + 159 unchanged - 5 fixed = 170 total (was 164)
          +1 mvnsite 2m 32s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          -1 findbugs 1m 46s hadoop-common-project/hadoop-common generated 5 new + 0 unchanged - 0 fixed = 5 total (was 0)
          +1 javadoc 2m 15s the patch passed
                Other Tests
          -1 unit 7m 22s hadoop-common in the patch failed.
          -1 unit 86m 27s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 40s The patch does not generate ASF License warnings.
          184m 20s



          Reason Tests
          FindBugs module:hadoop-common-project/hadoop-common
            Boxing/unboxing to parse a primitive org.apache.hadoop.fs.viewfs.NflyFSystem.createFileSystem(URI[], Configuration, String) At NflyFSystem.java:org.apache.hadoop.fs.viewfs.NflyFSystem.createFileSystem(URI[], Configuration, String) At NflyFSystem.java:[line 933]
            org.apache.hadoop.fs.viewfs.NflyFSystem$MRNflyNode doesn't override org.apache.hadoop.net.NodeBase.equals(Object) At NflyFSystem.java:At NflyFSystem.java:[line 1]
            org.apache.hadoop.fs.viewfs.NflyFSystem$NflyNode doesn't override org.apache.hadoop.net.NodeBase.equals(Object) At NflyFSystem.java:At NflyFSystem.java:[line 1]
            org.apache.hadoop.fs.viewfs.NflyFSystem$NflyStatus overrides equals in org.apache.hadoop.fs.FileStatus and may not be symmetric At NflyFSystem.java:and may not be symmetric At NflyFSystem.java:[lines 523-526]
            Class org.apache.hadoop.fs.viewfs.NflyFSystem$NflyStatus defines non-transient non-serializable instance field realFs In NflyFSystem.java:instance field realFs In NflyFSystem.java
          Failed junit tests hadoop.net.TestDNS
            hadoop.hdfs.TestDFSStripedOutputStreamWithFailure080



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:14b5c93
          JIRA Issue HADOOP-12077
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12880113/HADOOP-12077.006.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 3876c19b6643 3.13.0-117-generic #164-Ubuntu SMP Fri Apr 7 11:05:26 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / c5d256c
          Default Java 1.8.0_131
          findbugs v3.1.0-RC1
          findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/12942/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html
          javac https://builds.apache.org/job/PreCommit-HADOOP-Build/12942/artifact/patchprocess/diff-compile-javac-root.txt
          checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/12942/artifact/patchprocess/diff-checkstyle-root.txt
          findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/12942/artifact/patchprocess/new-findbugs-hadoop-common-project_hadoop-common.html
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/12942/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/12942/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/12942/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/12942/console
          Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 19s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.       trunk Compile Tests 0 mvndep 1m 27s Maven dependency ordering for branch +1 mvninstall 15m 15s trunk passed +1 compile 15m 57s trunk passed +1 checkstyle 2m 5s trunk passed +1 mvnsite 2m 34s trunk passed -1 findbugs 1m 54s hadoop-hdfs-project/hadoop-hdfs in trunk has 9 extant Findbugs warnings. +1 javadoc 1m 42s trunk passed       Patch Compile Tests 0 mvndep 0m 17s Maven dependency ordering for patch +1 mvninstall 1m 40s the patch passed +1 compile 12m 5s the patch passed -1 javac 12m 5s root generated 5 new + 1418 unchanged - 0 fixed = 1423 total (was 1418) -0 checkstyle 2m 2s root: The patch generated 11 new + 159 unchanged - 5 fixed = 170 total (was 164) +1 mvnsite 2m 32s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. -1 findbugs 1m 46s hadoop-common-project/hadoop-common generated 5 new + 0 unchanged - 0 fixed = 5 total (was 0) +1 javadoc 2m 15s the patch passed       Other Tests -1 unit 7m 22s hadoop-common in the patch failed. -1 unit 86m 27s hadoop-hdfs in the patch failed. +1 asflicense 0m 40s The patch does not generate ASF License warnings. 184m 20s Reason Tests FindBugs module:hadoop-common-project/hadoop-common   Boxing/unboxing to parse a primitive org.apache.hadoop.fs.viewfs.NflyFSystem.createFileSystem(URI[], Configuration, String) At NflyFSystem.java:org.apache.hadoop.fs.viewfs.NflyFSystem.createFileSystem(URI[], Configuration, String) At NflyFSystem.java: [line 933]   org.apache.hadoop.fs.viewfs.NflyFSystem$MRNflyNode doesn't override org.apache.hadoop.net.NodeBase.equals(Object) At NflyFSystem.java:At NflyFSystem.java: [line 1]   org.apache.hadoop.fs.viewfs.NflyFSystem$NflyNode doesn't override org.apache.hadoop.net.NodeBase.equals(Object) At NflyFSystem.java:At NflyFSystem.java: [line 1]   org.apache.hadoop.fs.viewfs.NflyFSystem$NflyStatus overrides equals in org.apache.hadoop.fs.FileStatus and may not be symmetric At NflyFSystem.java:and may not be symmetric At NflyFSystem.java: [lines 523-526]   Class org.apache.hadoop.fs.viewfs.NflyFSystem$NflyStatus defines non-transient non-serializable instance field realFs In NflyFSystem.java:instance field realFs In NflyFSystem.java Failed junit tests hadoop.net.TestDNS   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure080 Subsystem Report/Notes Docker Image:yetus/hadoop:14b5c93 JIRA Issue HADOOP-12077 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12880113/HADOOP-12077.006.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 3876c19b6643 3.13.0-117-generic #164-Ubuntu SMP Fri Apr 7 11:05:26 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / c5d256c Default Java 1.8.0_131 findbugs v3.1.0-RC1 findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/12942/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html javac https://builds.apache.org/job/PreCommit-HADOOP-Build/12942/artifact/patchprocess/diff-compile-javac-root.txt checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/12942/artifact/patchprocess/diff-checkstyle-root.txt findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/12942/artifact/patchprocess/new-findbugs-hadoop-common-project_hadoop-common.html unit https://builds.apache.org/job/PreCommit-HADOOP-Build/12942/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/12942/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/12942/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/12942/console Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          chris.douglas Chris Douglas added a comment -

          Fixed some warnings. The javac warning depends on HADOOP-13065, particularly HADOOP-13032.

          I didn't do any testing with this patch, I only rebased it and ran unit tests. Gera Shegalov, do you have cycles to verify that it works as designed?

          Show
          chris.douglas Chris Douglas added a comment - Fixed some warnings. The javac warning depends on HADOOP-13065 , particularly HADOOP-13032 . I didn't do any testing with this patch, I only rebased it and ran unit tests. Gera Shegalov , do you have cycles to verify that it works as designed?
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 21s Docker mode activated.
                Prechecks
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
                trunk Compile Tests
          0 mvndep 1m 25s Maven dependency ordering for branch
          +1 mvninstall 15m 14s trunk passed
          +1 compile 17m 7s trunk passed
          +1 checkstyle 2m 24s trunk passed
          +1 mvnsite 2m 52s trunk passed
          -1 findbugs 2m 2s hadoop-hdfs-project/hadoop-hdfs in trunk has 9 extant Findbugs warnings.
          +1 javadoc 2m 4s trunk passed
                Patch Compile Tests
          0 mvndep 0m 18s Maven dependency ordering for patch
          +1 mvninstall 1m 58s the patch passed
          +1 compile 15m 35s the patch passed
          -1 javac 15m 35s root generated 1 new + 1418 unchanged - 0 fixed = 1419 total (was 1418)
          -0 checkstyle 2m 16s root: The patch generated 12 new + 159 unchanged - 5 fixed = 171 total (was 164)
          +1 mvnsite 2m 51s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 4m 17s the patch passed
          +1 javadoc 2m 23s the patch passed
                Other Tests
          +1 unit 9m 14s hadoop-common in the patch passed.
          -1 unit 93m 28s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 37s The patch does not generate ASF License warnings.
          200m 56s



          Reason Tests
          Failed junit tests hadoop.hdfs.TestDFSStripedOutputStreamWithFailure070
            hadoop.hdfs.TestDFSStripedOutputStreamWithFailure080
            hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting
            hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:14b5c93
          JIRA Issue HADOOP-12077
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12880323/HADOOP-12077.007.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux e1654039e1a9 3.13.0-123-generic #172-Ubuntu SMP Mon Jun 26 18:04:35 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / f4c6b00
          Default Java 1.8.0_131
          findbugs v3.1.0-RC1
          findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/12949/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html
          javac https://builds.apache.org/job/PreCommit-HADOOP-Build/12949/artifact/patchprocess/diff-compile-javac-root.txt
          checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/12949/artifact/patchprocess/diff-checkstyle-root.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/12949/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/12949/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/12949/console
          Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 21s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.       trunk Compile Tests 0 mvndep 1m 25s Maven dependency ordering for branch +1 mvninstall 15m 14s trunk passed +1 compile 17m 7s trunk passed +1 checkstyle 2m 24s trunk passed +1 mvnsite 2m 52s trunk passed -1 findbugs 2m 2s hadoop-hdfs-project/hadoop-hdfs in trunk has 9 extant Findbugs warnings. +1 javadoc 2m 4s trunk passed       Patch Compile Tests 0 mvndep 0m 18s Maven dependency ordering for patch +1 mvninstall 1m 58s the patch passed +1 compile 15m 35s the patch passed -1 javac 15m 35s root generated 1 new + 1418 unchanged - 0 fixed = 1419 total (was 1418) -0 checkstyle 2m 16s root: The patch generated 12 new + 159 unchanged - 5 fixed = 171 total (was 164) +1 mvnsite 2m 51s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 4m 17s the patch passed +1 javadoc 2m 23s the patch passed       Other Tests +1 unit 9m 14s hadoop-common in the patch passed. -1 unit 93m 28s hadoop-hdfs in the patch failed. +1 asflicense 0m 37s The patch does not generate ASF License warnings. 200m 56s Reason Tests Failed junit tests hadoop.hdfs.TestDFSStripedOutputStreamWithFailure070   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure080   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure Subsystem Report/Notes Docker Image:yetus/hadoop:14b5c93 JIRA Issue HADOOP-12077 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12880323/HADOOP-12077.007.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux e1654039e1a9 3.13.0-123-generic #172-Ubuntu SMP Mon Jun 26 18:04:35 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / f4c6b00 Default Java 1.8.0_131 findbugs v3.1.0-RC1 findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/12949/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html javac https://builds.apache.org/job/PreCommit-HADOOP-Build/12949/artifact/patchprocess/diff-compile-javac-root.txt checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/12949/artifact/patchprocess/diff-checkstyle-root.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/12949/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/12949/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/12949/console Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          jira.shegalov Gera Shegalov added a comment -

          Chris Douglas, yes, I'll spend time to double-check. Ming Ma are you aware of any fixes from the internal branch we need to incorporate here or in a follow up JIRA?

          Show
          jira.shegalov Gera Shegalov added a comment - Chris Douglas , yes, I'll spend time to double-check. Ming Ma are you aware of any fixes from the internal branch we need to incorporate here or in a follow up JIRA?
          Hide
          mingma Ming Ma added a comment -

          There are couple fixes either we need or have been committed in open source.

          • Make Trash work for nfly. Given our internal patch requires changes in FileSystem APIs and thus likely requires more discussion, I recommend opening a separate jira.
          • Fix the cross-DC write performance issue. HDFS-9259 is already in upstream.
          • Pick the closest FileSystem as the preferred location for read. All required fixes are already in upstream. Specifically it needs HDFS-10206 and the change in NetworkTopology class from HDFS-7647.

          Thus this patch as it is has covered the main functionality. We can commit this patch after fixing checkstyle etc and then use separate jiras for follow-up for major issues. Thoughts?

          Show
          mingma Ming Ma added a comment - There are couple fixes either we need or have been committed in open source. Make Trash work for nfly. Given our internal patch requires changes in FileSystem APIs and thus likely requires more discussion, I recommend opening a separate jira. Fix the cross-DC write performance issue. HDFS-9259 is already in upstream. Pick the closest FileSystem as the preferred location for read. All required fixes are already in upstream. Specifically it needs HDFS-10206 and the change in NetworkTopology class from HDFS-7647 . Thus this patch as it is has covered the main functionality. We can commit this patch after fixing checkstyle etc and then use separate jiras for follow-up for major issues. Thoughts?
          Hide
          chris.douglas Chris Douglas added a comment -

          this patch as it is has covered the main functionality. We can commit this patch after fixing checkstyle etc and then use separate jiras for follow-up for major issues. Thoughts?

          The unit tests around ViewFS are pretty thorough, so I'm reasonably sure this won't cause a regression. We can handle the other issues as followup. Would it be difficult to verify that basic operations work as expected in your environment?

          Filed HADOOP-14742 to update the ViewFS docs.

          Show
          chris.douglas Chris Douglas added a comment - this patch as it is has covered the main functionality. We can commit this patch after fixing checkstyle etc and then use separate jiras for follow-up for major issues. Thoughts? The unit tests around ViewFS are pretty thorough, so I'm reasonably sure this won't cause a regression. We can handle the other issues as followup. Would it be difficult to verify that basic operations work as expected in your environment? Filed HADOOP-14742 to update the ViewFS docs .
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 15s Docker mode activated.
                Prechecks
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
                trunk Compile Tests
          0 mvndep 0m 15s Maven dependency ordering for branch
          +1 mvninstall 14m 58s trunk passed
          +1 compile 17m 58s trunk passed
          +1 checkstyle 3m 17s trunk passed
          +1 mvnsite 3m 5s trunk passed
          -1 findbugs 2m 16s hadoop-hdfs-project/hadoop-hdfs in trunk has 9 extant Findbugs warnings.
          +1 javadoc 2m 4s trunk passed
                Patch Compile Tests
          0 mvndep 0m 33s Maven dependency ordering for patch
          -1 mvninstall 1m 12s hadoop-hdfs in the patch failed.
          +1 compile 13m 39s the patch passed
          -1 javac 13m 39s root generated 1 new + 1379 unchanged - 0 fixed = 1380 total (was 1379)
          -0 checkstyle 2m 1s root: The patch generated 1 new + 154 unchanged - 9 fixed = 155 total (was 163)
          +1 mvnsite 2m 32s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 3m 36s the patch passed
          +1 javadoc 1m 46s the patch passed
                Other Tests
          -1 unit 8m 9s hadoop-common in the patch failed.
          -1 unit 65m 16s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 36s The patch does not generate ASF License warnings.
          168m 41s



          Reason Tests
          Failed junit tests hadoop.security.TestKDiag
            hadoop.hdfs.TestRollingUpgrade
            hadoop.hdfs.TestDFSStripedOutputStreamWithFailure150
            hadoop.hdfs.TestDFSStripedOutputStreamWithFailure080
            hadoop.hdfs.TestDFSStripedOutputStreamWithFailure140
            hadoop.hdfs.server.namenode.ha.TestPipelinesFailover



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:14b5c93
          JIRA Issue HADOOP-12077
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12880734/HADOOP-12077.008.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 12dfaeb63f74 3.13.0-117-generic #164-Ubuntu SMP Fri Apr 7 11:05:26 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / bc20680
          Default Java 1.8.0_131
          findbugs v3.1.0-RC1
          findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/12973/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html
          mvninstall https://builds.apache.org/job/PreCommit-HADOOP-Build/12973/artifact/patchprocess/patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs.txt
          javac https://builds.apache.org/job/PreCommit-HADOOP-Build/12973/artifact/patchprocess/diff-compile-javac-root.txt
          checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/12973/artifact/patchprocess/diff-checkstyle-root.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/12973/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/12973/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/12973/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/12973/console
          Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 15s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.       trunk Compile Tests 0 mvndep 0m 15s Maven dependency ordering for branch +1 mvninstall 14m 58s trunk passed +1 compile 17m 58s trunk passed +1 checkstyle 3m 17s trunk passed +1 mvnsite 3m 5s trunk passed -1 findbugs 2m 16s hadoop-hdfs-project/hadoop-hdfs in trunk has 9 extant Findbugs warnings. +1 javadoc 2m 4s trunk passed       Patch Compile Tests 0 mvndep 0m 33s Maven dependency ordering for patch -1 mvninstall 1m 12s hadoop-hdfs in the patch failed. +1 compile 13m 39s the patch passed -1 javac 13m 39s root generated 1 new + 1379 unchanged - 0 fixed = 1380 total (was 1379) -0 checkstyle 2m 1s root: The patch generated 1 new + 154 unchanged - 9 fixed = 155 total (was 163) +1 mvnsite 2m 32s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 3m 36s the patch passed +1 javadoc 1m 46s the patch passed       Other Tests -1 unit 8m 9s hadoop-common in the patch failed. -1 unit 65m 16s hadoop-hdfs in the patch failed. +1 asflicense 0m 36s The patch does not generate ASF License warnings. 168m 41s Reason Tests Failed junit tests hadoop.security.TestKDiag   hadoop.hdfs.TestRollingUpgrade   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure150   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure080   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure140   hadoop.hdfs.server.namenode.ha.TestPipelinesFailover Subsystem Report/Notes Docker Image:yetus/hadoop:14b5c93 JIRA Issue HADOOP-12077 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12880734/HADOOP-12077.008.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 12dfaeb63f74 3.13.0-117-generic #164-Ubuntu SMP Fri Apr 7 11:05:26 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / bc20680 Default Java 1.8.0_131 findbugs v3.1.0-RC1 findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/12973/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html mvninstall https://builds.apache.org/job/PreCommit-HADOOP-Build/12973/artifact/patchprocess/patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs.txt javac https://builds.apache.org/job/PreCommit-HADOOP-Build/12973/artifact/patchprocess/diff-compile-javac-root.txt checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/12973/artifact/patchprocess/diff-checkstyle-root.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/12973/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/12973/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/12973/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/12973/console Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 17s Docker mode activated.
                Prechecks
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
                trunk Compile Tests
          0 mvndep 0m 20s Maven dependency ordering for branch
          +1 mvninstall 13m 42s trunk passed
          +1 compile 14m 54s trunk passed
          +1 checkstyle 1m 54s trunk passed
          +1 mvnsite 2m 29s trunk passed
          -1 findbugs 1m 46s hadoop-hdfs-project/hadoop-hdfs in trunk has 9 extant Findbugs warnings.
          +1 javadoc 1m 32s trunk passed
                Patch Compile Tests
          0 mvndep 0m 16s Maven dependency ordering for patch
          +1 mvninstall 1m 30s the patch passed
          +1 compile 10m 53s the patch passed
          -1 javac 10m 53s root generated 1 new + 1377 unchanged - 0 fixed = 1378 total (was 1377)
          +1 checkstyle 2m 28s root: The patch generated 0 new + 153 unchanged - 11 fixed = 153 total (was 164)
          +1 mvnsite 2m 35s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 3m 29s the patch passed
          +1 javadoc 1m 33s the patch passed
                Other Tests
          -1 unit 7m 18s hadoop-common in the patch failed.
          -1 unit 77m 16s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 30s The patch does not generate ASF License warnings.
          168m 23s



          Reason Tests
          Failed junit tests hadoop.net.TestDNS
            hadoop.hdfs.TestDFSStripedInputStreamWithRandomECPolicy
            hadoop.hdfs.TestDFSStripedOutputStreamWithFailure140
            hadoop.hdfs.server.blockmanagement.TestRBWBlockInvalidation
            hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting
            hadoop.hdfs.web.TestWebHdfsTimeouts



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:14b5c93
          JIRA Issue HADOOP-12077
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12880927/HADOOP-12077.009.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 57190f6925a5 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12 13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / f4e1aa0
          Default Java 1.8.0_131
          findbugs v3.1.0-RC1
          findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/12988/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html
          javac https://builds.apache.org/job/PreCommit-HADOOP-Build/12988/artifact/patchprocess/diff-compile-javac-root.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/12988/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/12988/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/12988/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/12988/console
          Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 17s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.       trunk Compile Tests 0 mvndep 0m 20s Maven dependency ordering for branch +1 mvninstall 13m 42s trunk passed +1 compile 14m 54s trunk passed +1 checkstyle 1m 54s trunk passed +1 mvnsite 2m 29s trunk passed -1 findbugs 1m 46s hadoop-hdfs-project/hadoop-hdfs in trunk has 9 extant Findbugs warnings. +1 javadoc 1m 32s trunk passed       Patch Compile Tests 0 mvndep 0m 16s Maven dependency ordering for patch +1 mvninstall 1m 30s the patch passed +1 compile 10m 53s the patch passed -1 javac 10m 53s root generated 1 new + 1377 unchanged - 0 fixed = 1378 total (was 1377) +1 checkstyle 2m 28s root: The patch generated 0 new + 153 unchanged - 11 fixed = 153 total (was 164) +1 mvnsite 2m 35s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 3m 29s the patch passed +1 javadoc 1m 33s the patch passed       Other Tests -1 unit 7m 18s hadoop-common in the patch failed. -1 unit 77m 16s hadoop-hdfs in the patch failed. +1 asflicense 0m 30s The patch does not generate ASF License warnings. 168m 23s Reason Tests Failed junit tests hadoop.net.TestDNS   hadoop.hdfs.TestDFSStripedInputStreamWithRandomECPolicy   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure140   hadoop.hdfs.server.blockmanagement.TestRBWBlockInvalidation   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting   hadoop.hdfs.web.TestWebHdfsTimeouts Subsystem Report/Notes Docker Image:yetus/hadoop:14b5c93 JIRA Issue HADOOP-12077 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12880927/HADOOP-12077.009.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 57190f6925a5 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12 13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / f4e1aa0 Default Java 1.8.0_131 findbugs v3.1.0-RC1 findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/12988/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html javac https://builds.apache.org/job/PreCommit-HADOOP-Build/12988/artifact/patchprocess/diff-compile-javac-root.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/12988/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/12988/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/12988/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/12988/console Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          chris.douglas Chris Douglas added a comment -

          Gera Shegalov, Ming Ma Have you had a chance to test the patch?

          Show
          chris.douglas Chris Douglas added a comment - Gera Shegalov , Ming Ma Have you had a chance to test the patch?
          Hide
          jira.shegalov Gera Shegalov added a comment -

          Hi Chris Douglas, I just came back from vacation. I'll test everything I can as unprivileged user and will talk to Ming Ma if I need some root functionality.

          Show
          jira.shegalov Gera Shegalov added a comment - Hi Chris Douglas , I just came back from vacation. I'll test everything I can as unprivileged user and will talk to Ming Ma if I need some root functionality.
          Hide
          jira.shegalov Gera Shegalov added a comment -

          Got the basic setup and the read path working without code modifications with the trunk version of the patch HADOOP-12077.009 just by tweaking our configs against our 2.6+patches clusters. On the write path I need to check rename.

          java.lang.ClassCastException: org.apache.hadoop.fs.viewfs.NflyFSystem cannot be cast to org.apache.hadoop.fs.viewfs.ChRootedFileSystem
                  at org.apache.hadoop.fs.viewfs.ViewFileSystem.rename(ViewFileSystem.java:504)
                  at org.apache.hadoop.fs.FilterFileSystem.rename(FilterFileSystem.java:235)
                  at org.apache.hadoop.fs.shell.CommandWithDestination$TargetFileSystem.rename(CommandWithDestination.java:526)
                  at org.apache.hadoop.fs.shell.CommandWithDestination.copyStreamToTarget(CommandWithDestination.java:409)
                  at org.apache.hadoop.fs.shell.CopyCommands$Put.processArguments(CopyCommands.java:287)
                  at org.apache.hadoop.fs.shell.FsCommand.processRawArguments(FsCommand.java:119)
                  at org.apache.hadoop.fs.shell.Command.run(Command.java:176)
                  at org.apache.hadoop.fs.FsShell.run(FsShell.java:326)
                  at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
                  at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90)
                  at org.apache.hadoop.fs.FsShell.main(FsShell.java:389)
          
          Show
          jira.shegalov Gera Shegalov added a comment - Got the basic setup and the read path working without code modifications with the trunk version of the patch HADOOP-12077 .009 just by tweaking our configs against our 2.6+patches clusters. On the write path I need to check rename. java.lang.ClassCastException: org.apache.hadoop.fs.viewfs.NflyFSystem cannot be cast to org.apache.hadoop.fs.viewfs.ChRootedFileSystem at org.apache.hadoop.fs.viewfs.ViewFileSystem.rename(ViewFileSystem.java:504) at org.apache.hadoop.fs.FilterFileSystem.rename(FilterFileSystem.java:235) at org.apache.hadoop.fs.shell.CommandWithDestination$TargetFileSystem.rename(CommandWithDestination.java:526) at org.apache.hadoop.fs.shell.CommandWithDestination.copyStreamToTarget(CommandWithDestination.java:409) at org.apache.hadoop.fs.shell.CopyCommands$Put.processArguments(CopyCommands.java:287) at org.apache.hadoop.fs.shell.FsCommand.processRawArguments(FsCommand.java:119) at org.apache.hadoop.fs.shell.Command.run(Command.java:176) at org.apache.hadoop.fs.FsShell.run(FsShell.java:326) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90) at org.apache.hadoop.fs.FsShell.main(FsShell.java:389)
          Hide
          jira.shegalov Gera Shegalov added a comment -

          Chris Douglas, I added a test for the regression and verified the patch in our environment.

          Show
          jira.shegalov Gera Shegalov added a comment - Chris Douglas , I added a test for the regression and verified the patch in our environment.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 17s Docker mode activated.
                Prechecks
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
                trunk Compile Tests
          0 mvndep 0m 45s Maven dependency ordering for branch
          +1 mvninstall 13m 41s trunk passed
          +1 compile 14m 31s trunk passed
          +1 checkstyle 2m 0s trunk passed
          +1 mvnsite 2m 1s trunk passed
          +1 findbugs 3m 15s trunk passed
          +1 javadoc 1m 42s trunk passed
                Patch Compile Tests
          0 mvndep 0m 16s Maven dependency ordering for patch
          +1 mvninstall 1m 34s the patch passed
          +1 compile 10m 57s the patch passed
          -1 javac 10m 57s root generated 1 new + 1285 unchanged - 0 fixed = 1286 total (was 1285)
          -0 checkstyle 2m 5s root: The patch generated 2 new + 152 unchanged - 11 fixed = 154 total (was 163)
          +1 mvnsite 2m 5s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 3m 37s the patch passed
          +1 javadoc 1m 51s the patch passed
                Other Tests
          -1 unit 8m 8s hadoop-common in the patch failed.
          -1 unit 94m 4s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 39s The patch does not generate ASF License warnings.
          185m 40s



          Reason Tests
          Failed junit tests hadoop.security.TestRaceWhenRelogin
            hadoop.security.TestKDiag
            hadoop.hdfs.TestBlockStoragePolicy
            hadoop.hdfs.TestClientProtocolForPipelineRecovery
            hadoop.hdfs.TestLeaseRecoveryStriped
            hadoop.hdfs.TestDFSStripedOutputStreamWithFailure200
            hadoop.hdfs.TestDFSStripedOutputStreamWithFailure110
            hadoop.hdfs.TestReplaceDatanodeOnFailure
            hadoop.hdfs.TestReadStripedFileWithDecoding
            hadoop.hdfs.TestEncryptionZonesWithHA
            hadoop.hdfs.TestDFSStripedOutputStreamWithFailure140
            hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting
            hadoop.hdfs.TestFileAppendRestart
          Timed out junit tests org.apache.hadoop.hdfs.TestWriteReadStripedFile



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:71bbb86
          JIRA Issue HADOOP-12077
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12885126/HADOOP-12077.010.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 248e8a593816 3.13.0-117-generic #164-Ubuntu SMP Fri Apr 7 11:05:26 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 275980b
          Default Java 1.8.0_144
          findbugs v3.1.0-RC1
          javac https://builds.apache.org/job/PreCommit-HADOOP-Build/13158/artifact/patchprocess/diff-compile-javac-root.txt
          checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/13158/artifact/patchprocess/diff-checkstyle-root.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/13158/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/13158/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/13158/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/13158/console
          Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 17s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.       trunk Compile Tests 0 mvndep 0m 45s Maven dependency ordering for branch +1 mvninstall 13m 41s trunk passed +1 compile 14m 31s trunk passed +1 checkstyle 2m 0s trunk passed +1 mvnsite 2m 1s trunk passed +1 findbugs 3m 15s trunk passed +1 javadoc 1m 42s trunk passed       Patch Compile Tests 0 mvndep 0m 16s Maven dependency ordering for patch +1 mvninstall 1m 34s the patch passed +1 compile 10m 57s the patch passed -1 javac 10m 57s root generated 1 new + 1285 unchanged - 0 fixed = 1286 total (was 1285) -0 checkstyle 2m 5s root: The patch generated 2 new + 152 unchanged - 11 fixed = 154 total (was 163) +1 mvnsite 2m 5s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 3m 37s the patch passed +1 javadoc 1m 51s the patch passed       Other Tests -1 unit 8m 8s hadoop-common in the patch failed. -1 unit 94m 4s hadoop-hdfs in the patch failed. +1 asflicense 0m 39s The patch does not generate ASF License warnings. 185m 40s Reason Tests Failed junit tests hadoop.security.TestRaceWhenRelogin   hadoop.security.TestKDiag   hadoop.hdfs.TestBlockStoragePolicy   hadoop.hdfs.TestClientProtocolForPipelineRecovery   hadoop.hdfs.TestLeaseRecoveryStriped   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure200   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure110   hadoop.hdfs.TestReplaceDatanodeOnFailure   hadoop.hdfs.TestReadStripedFileWithDecoding   hadoop.hdfs.TestEncryptionZonesWithHA   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure140   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting   hadoop.hdfs.TestFileAppendRestart Timed out junit tests org.apache.hadoop.hdfs.TestWriteReadStripedFile Subsystem Report/Notes Docker Image:yetus/hadoop:71bbb86 JIRA Issue HADOOP-12077 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12885126/HADOOP-12077.010.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 248e8a593816 3.13.0-117-generic #164-Ubuntu SMP Fri Apr 7 11:05:26 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 275980b Default Java 1.8.0_144 findbugs v3.1.0-RC1 javac https://builds.apache.org/job/PreCommit-HADOOP-Build/13158/artifact/patchprocess/diff-compile-javac-root.txt checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/13158/artifact/patchprocess/diff-checkstyle-root.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/13158/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/13158/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/13158/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/13158/console Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          chris.douglas Chris Douglas added a comment -

          I went through the test failures; they appear unrelated. The javac warning is blocked on related JIRAs. I can fix the checkstyle warnings before commit. If there are no objections, I'll commit this in a day or two.

          Show
          chris.douglas Chris Douglas added a comment - I went through the test failures; they appear unrelated. The javac warning is blocked on related JIRAs. I can fix the checkstyle warnings before commit. If there are no objections, I'll commit this in a day or two.
          Hide
          chris.douglas Chris Douglas added a comment -

          +1 I committed this. Thanks Gera

          Show
          chris.douglas Chris Douglas added a comment - +1 I committed this. Thanks Gera
          Hide
          jira.shegalov Gera Shegalov added a comment -

          Thanks Chris Douglas for taking care of fixing up the patch, and committing it

          Show
          jira.shegalov Gera Shegalov added a comment - Thanks Chris Douglas for taking care of fixing up the patch, and committing it
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #12811 (See https://builds.apache.org/job/Hadoop-trunk-Commit/12811/)
          HADOOP-12077. Provide a multi-URI replication Inode for ViewFs. (cdouglas: rev 1f3bc63e6772be81bc9a6a7d93ed81d2a9e066c0)

          • (add) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/NflyFSystem.java
          • (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ViewFs.java
          • (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/Constants.java
          • (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/viewfs/TestViewFsConfig.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/fs/viewfs/TestViewFileSystemHdfs.java
          • (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/InodeTree.java
          • (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/viewfs/TestViewFileSystemLocalFileSystem.java
          • (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ViewFileSystem.java
          • (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ConfigUtil.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #12811 (See https://builds.apache.org/job/Hadoop-trunk-Commit/12811/ ) HADOOP-12077 . Provide a multi-URI replication Inode for ViewFs. (cdouglas: rev 1f3bc63e6772be81bc9a6a7d93ed81d2a9e066c0) (add) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/NflyFSystem.java (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ViewFs.java (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/Constants.java (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/viewfs/TestViewFsConfig.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/fs/viewfs/TestViewFileSystemHdfs.java (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/InodeTree.java (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/viewfs/TestViewFileSystemLocalFileSystem.java (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ViewFileSystem.java (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ConfigUtil.java

            People

            • Assignee:
              jira.shegalov Gera Shegalov
              Reporter:
              jira.shegalov Gera Shegalov
            • Votes:
              1 Vote for this issue
              Watchers:
              40 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development