Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-13363

Upgrade protobuf from 2.5.0 to something newer

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 3.0.0-alpha1, 3.0.0-alpha2
    • Fix Version/s: None
    • Component/s: build
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Incompatible change

      Description

      Standard protobuf 2.5.0 does not work properly on many platforms. (See, for example, https://gist.github.com/BennettSmith/7111094 ). In order for us to avoid crazy work arounds in the build environment and the fact that 2.5.0 is starting to slowly disappear as a standard install-able package for even Linux/x86, we need to either upgrade or self bundle or something else.

      1. HADOOP-13363.005.patch
        2 kB
        Tsuyoshi Ozawa
      2. HADOOP-13363.004.patch
        2 kB
        Tsuyoshi Ozawa
      3. HADOOP-13363.003.patch
        1 kB
        Tsuyoshi Ozawa
      4. HADOOP-13363.002.patch
        1 kB
        Tsuyoshi Ozawa
      5. HADOOP-13363.001.patch
        0.5 kB
        Tsuyoshi Ozawa

        Issue Links

          Activity

          Hide
          stack stack added a comment -

          I'd presume upgrade but run in pb2 mode, the default. That has been wire compat in my limited experience. We could do a bit of testing I suppose.

          Show
          stack stack added a comment - I'd presume upgrade but run in pb2 mode, the default. That has been wire compat in my limited experience. We could do a bit of testing I suppose.
          Hide
          ozawa Tsuyoshi Ozawa added a comment -

          We need wire-compatibility no matter when we upgrade, whether it's from 2.x to 3.0, or 2.x to 3.1.

          How can we ensure it? My first thought to ensure this is to run cluster with protobuf 3.x cluster jar and clients with protobuf 2.x jar.

          By the way, the current latest version of protobuf seems to be v3.4.0.
          https://github.com/google/protobuf/releases

          Show
          ozawa Tsuyoshi Ozawa added a comment - We need wire-compatibility no matter when we upgrade, whether it's from 2.x to 3.0, or 2.x to 3.1. How can we ensure it? My first thought to ensure this is to run cluster with protobuf 3.x cluster jar and clients with protobuf 2.x jar. By the way, the current latest version of protobuf seems to be v3.4.0. https://github.com/google/protobuf/releases
          Hide
          aw Allen Wittenauer added a comment -

          Also, the motivation I've heard for upgrading is to make it easier for devs to build Hadoop since it's hard to find a copy of PB 2.5.0.

          That's one issue, but it's not the only one. See the description of this very jira.

          Show
          aw Allen Wittenauer added a comment - Also, the motivation I've heard for upgrading is to make it easier for devs to build Hadoop since it's hard to find a copy of PB 2.5.0. That's one issue, but it's not the only one. See the description of this very jira.
          Hide
          andrew.wang Andrew Wang added a comment -

          I consider shading a prerequisite for the PB upgrade. If we shade, there aren't any API/ABI compatibility issues. We need wire-compatibility no matter when we upgrade, whether it's from 2.x to 3.0, or 2.x to 3.1.

          Also, the motivation I've heard for upgrading is to make it easier for devs to build Hadoop since it's hard to find a copy of PB 2.5.0. If that's the problem, there are other solutions that do not have any downstream user impact. For instance, hosting the PB 2.5.0 binary somewhere, or providing it in a docker image.

          Show
          andrew.wang Andrew Wang added a comment - I consider shading a prerequisite for the PB upgrade. If we shade, there aren't any API/ABI compatibility issues. We need wire-compatibility no matter when we upgrade, whether it's from 2.x to 3.0, or 2.x to 3.1. Also, the motivation I've heard for upgrading is to make it easier for devs to build Hadoop since it's hard to find a copy of PB 2.5.0. If that's the problem, there are other solutions that do not have any downstream user impact. For instance, hosting the PB 2.5.0 binary somewhere, or providing it in a docker image.
          Hide
          chris.douglas Chris Douglas added a comment -

          I still remember when we bumped the protobuf version shortly before 2.2.0 GA, and the downstream pain it caused due to ABI conflicts

          We certainly want to avoid that, but does anyone have cycles to fit this between 3.0.0-beta1 and GA? Bumping the PB version in a minor release would be regrettable, particularly since we're making such a big jump. Taking the hit in 3.0.0 and addressing incompatibility in a minor, even patch release might be easier.

          Users expect upgrading to a major version to require some additional testing. Bumping this in a minor version might cause more downstream headaches, even if careful shading would avoid more actual incompatibilities. Most of our use of PB is behind interfaces, so most users shouldn't notice...

          Show
          chris.douglas Chris Douglas added a comment - I still remember when we bumped the protobuf version shortly before 2.2.0 GA, and the downstream pain it caused due to ABI conflicts We certainly want to avoid that, but does anyone have cycles to fit this between 3.0.0-beta1 and GA? Bumping the PB version in a minor release would be regrettable, particularly since we're making such a big jump. Taking the hit in 3.0.0 and addressing incompatibility in a minor, even patch release might be easier. Users expect upgrading to a major version to require some additional testing. Bumping this in a minor version might cause more downstream headaches, even if careful shading would avoid more actual incompatibilities. Most of our use of PB is behind interfaces, so most users shouldn't notice...
          Hide
          andrew.wang Andrew Wang added a comment -

          Yes, basically. I still remember when we bumped the protobuf version shortly before 2.2.0 GA, and the downstream pain it caused due to ABI conflicts.

          We have the shaded-client now, but no one is using it yet, and there are also all the downstreams incorrectly depending on hadoop-hdfs rather than a client artifact. HBase was able to upgrade safely through shading PB. We also need a safe transition for our downstreams.

          Show
          andrew.wang Andrew Wang added a comment - Yes, basically. I still remember when we bumped the protobuf version shortly before 2.2.0 GA, and the downstream pain it caused due to ABI conflicts. We have the shaded-client now, but no one is using it yet, and there are also all the downstreams incorrectly depending on hadoop-hdfs rather than a client artifact. HBase was able to upgrade safely through shading PB. We also need a safe transition for our downstreams.
          Hide
          chris.douglas Chris Douglas added a comment -

          Where did this stall? On concerns that clients will need to rebuild because they link against the protobuf 2.5 jar, but not the 3.x jar?

          Show
          chris.douglas Chris Douglas added a comment - Where did this stall? On concerns that clients will need to rebuild because they link against the protobuf 2.5 jar, but not the 3.x jar?
          Hide
          andrew.wang Andrew Wang added a comment -

          I'm dropping this out of beta1 since it seems unlikely to happen in time. Given that we have client shading, we can revisit this later in a minor release.

          Show
          andrew.wang Andrew Wang added a comment - I'm dropping this out of beta1 since it seems unlikely to happen in time. Given that we have client shading, we can revisit this later in a minor release.
          Hide
          ozawa Tsuyoshi Ozawa added a comment -

          I will be back here after HADOOP-14284.

          Show
          ozawa Tsuyoshi Ozawa added a comment - I will be back here after HADOOP-14284 .
          Hide
          ozawa Tsuyoshi Ozawa added a comment -

          Make sense, thanks for clarification. It sounds awful.

          Let me try to create shaded-3rdparty-libs. I will try to shade both Guava and protobuf there.

          Show
          ozawa Tsuyoshi Ozawa added a comment - Make sense, thanks for clarification. It sounds awful. Let me try to create shaded-3rdparty-libs. I will try to shade both Guava and protobuf there.
          Hide
          stack stack added a comment -

          So, this would be squashing the above suggested 3rdparty-lib module instead into hadoop-common? It has some pluses. Here's a few downsides:

          • Everywhere EXCEPT in hadoop-common we'd have to change com.google.protobuf.* to instead point to the relocated classes bundled up in hadoop-common. This would be easy to do but would probably confuse developers (com.google.protobuf when in hadoop-common but org.apache.hadoop.shaded.com.google.protobuf.* everywhere else).
          • In your IDE, working on hadoop-common, you'd be fine but when working on a downstream module – say hdfs – you'd need to depend on a built hadoop-common artifact (unless we included src code for relocated classes).

          Pity there is not a pre-build. We make up an artifact of all relocated classes and push it to the repo. Then the main build would depend on it wth all hadoop references amended to refer to the relocated classes.

          Show
          stack stack added a comment - So, this would be squashing the above suggested 3rdparty-lib module instead into hadoop-common? It has some pluses. Here's a few downsides: Everywhere EXCEPT in hadoop-common we'd have to change com.google.protobuf.* to instead point to the relocated classes bundled up in hadoop-common. This would be easy to do but would probably confuse developers (com.google.protobuf when in hadoop-common but org.apache.hadoop.shaded.com.google.protobuf.* everywhere else). In your IDE, working on hadoop-common, you'd be fine but when working on a downstream module – say hdfs – you'd need to depend on a built hadoop-common artifact (unless we included src code for relocated classes). Pity there is not a pre-build. We make up an artifact of all relocated classes and push it to the repo. Then the main build would depend on it wth all hadoop references amended to refer to the relocated classes.
          Hide
          ozawa Tsuyoshi Ozawa added a comment -

          stack Thanks for your comment. After playing with Guava's problem for a while, I'm thinking that it's enough to shade protobuf/guava in hadoop-common because other components depend on common. What do you think? I think this workaround works well.

          Show
          ozawa Tsuyoshi Ozawa added a comment - stack Thanks for your comment. After playing with Guava's problem for a while, I'm thinking that it's enough to shade protobuf/guava in hadoop-common because other components depend on common. What do you think? I think this workaround works well.
          Hide
          stack stack added a comment -

          You can ask shading to only bundle required classes which should cut down on some of the duplicates and while I know we're not averse to carrying around a bit of fat in these parts, it does seem like we should try and avoid repeating pb classes seven times (counting above listed artifacts), at least.

          How you thinking of shading guava?

          stack could you turn over this issue?

          I'm a bit stuck at mo for spare cycles....

          Show
          stack stack added a comment - You can ask shading to only bundle required classes which should cut down on some of the duplicates and while I know we're not averse to carrying around a bit of fat in these parts, it does seem like we should try and avoid repeating pb classes seven times (counting above listed artifacts), at least. How you thinking of shading guava? stack could you turn over this issue? I'm a bit stuck at mo for spare cycles....
          Hide
          ozawa Tsuyoshi Ozawa added a comment - - edited

          Maybe I have not enough time to finish this until May 15th, to finish Guava related works until May 15th.

          stack could you turn over this issue?

          Show
          ozawa Tsuyoshi Ozawa added a comment - - edited Maybe I have not enough time to finish this until May 15th, to finish Guava related works until May 15th. stack could you turn over this issue?
          Hide
          ozawa Tsuyoshi Ozawa added a comment -

          Andrew Wang I thought this should be in alpha3, personally. After discussion on ML, we confirmed that proto2 syntax is compatible with protobuf spec v2. We have enough confidence to update protobuf version, I think.

          Would it make sense having a shaded-3rdparty-libs jar that had relocated protobuf and other libs we want to shade?

          It makes sense to me, but IIUC, this is one kind of optimization to reduce jar size. IMHO, we can make this change later. What do you think?

          Show
          ozawa Tsuyoshi Ozawa added a comment - Andrew Wang I thought this should be in alpha3, personally. After discussion on ML, we confirmed that proto2 syntax is compatible with protobuf spec v2. We have enough confidence to update protobuf version, I think. Would it make sense having a shaded-3rdparty-libs jar that had relocated protobuf and other libs we want to shade? It makes sense to me, but IIUC, this is one kind of optimization to reduce jar size. IMHO, we can make this change later. What do you think?
          Hide
          stack stack added a comment -

          Kihwal Lee Good point.

          What we thinking for a shading approach?

          Shading runs post package and pulls in the relocated classes into the jar that references them.

          I see hadoop common, hdfs, hdfs-client, mr-client, yarn-common, yarn-client, yarn-server all making reference to pb (I didn't look at src/test). We thinking we'd dupe instances of pb jars up into each of these artifacts.

          Would it make sense having a shaded-3rdparty-libs jar that had relocated protobuf and other libs we want to shade? Downside is all pb references would have to be changed to reference the relocated classes (org.apache.hadoop.com.google.protobuf.*).

          Would hadoop3 ship a pb2.5 at all?

          Show
          stack stack added a comment - Kihwal Lee Good point. What we thinking for a shading approach? Shading runs post package and pulls in the relocated classes into the jar that references them. I see hadoop common, hdfs, hdfs-client, mr-client, yarn-common, yarn-client, yarn-server all making reference to pb (I didn't look at src/test). We thinking we'd dupe instances of pb jars up into each of these artifacts. Would it make sense having a shaded-3rdparty-libs jar that had relocated protobuf and other libs we want to shade? Downside is all pb references would have to be changed to reference the relocated classes (org.apache.hadoop.com.google.protobuf.*). Would hadoop3 ship a pb2.5 at all?
          Hide
          andrew.wang Andrew Wang added a comment -

          Upgrading to blocker and setting target version for tracking purposes.

          Show
          andrew.wang Andrew Wang added a comment - Upgrading to blocker and setting target version for tracking purposes.
          Hide
          andrew.wang Andrew Wang added a comment -

          Kihwal Lee thanks for bringing this up. I believe this scenario will be covered by our existing fsimage upgrade tests, which run on checked-in fsimages.

          Hi Tsuyoshi Ozawa, what's the schedule for getting this committed? I'd like alpha3 to be our last alpha, and this is not a change I want to sneak in between alpha3 and beta1. The current planned date for alpha3 is May 15th.

          Also to be very explicit, I consider shading PB everywhere to be a prerequisite for upgrading to PB3.

          Show
          andrew.wang Andrew Wang added a comment - Kihwal Lee thanks for bringing this up. I believe this scenario will be covered by our existing fsimage upgrade tests, which run on checked-in fsimages. Hi Tsuyoshi Ozawa , what's the schedule for getting this committed? I'd like alpha3 to be our last alpha, and this is not a change I want to sneak in between alpha3 and beta1. The current planned date for alpha3 is May 15th. Also to be very explicit, I consider shading PB everywhere to be a prerequisite for upgrading to PB3.
          Hide
          kihwal Kihwal Lee added a comment -

          I am guessing the wire compatibility will cover this, but want to make sure. We are using protobuf to serialize hdfs fsimage out to disk. The on-disk data format should be compatible.

          Show
          kihwal Kihwal Lee added a comment - I am guessing the wire compatibility will cover this, but want to make sure. We are using protobuf to serialize hdfs fsimage out to disk. The on-disk data format should be compatible.
          Hide
          stack stack added a comment -

          Tsuyoshi Ozawa Yes sir. Default mode is proto2. You have to explicitly ask for proto3. When protoc runs, it emits a WARNING that the .proto does not have an explicit version designation but it is just a WARNING that can be quelled by explicitly stating protoc2 (we did not bother doing this in our project – not yet at least).

          Show
          stack stack added a comment - Tsuyoshi Ozawa Yes sir. Default mode is proto2. You have to explicitly ask for proto3. When protoc runs, it emits a WARNING that the .proto does not have an explicit version designation but it is just a WARNING that can be quelled by explicitly stating protoc2 (we did not bother doing this in our project – not yet at least).
          Hide
          andrew.wang Andrew Wang added a comment -

          Based on my read, proto2 is the default parsing mode, we need to explicitly opt-in to proto3. This is nice, since we won't have to modify our existing proto files if we upgrade PB.

          I think the mode also only affects parsing of the .proto file. proto3 adds new types like maps and unions, which aren't available in proto2 mode. I think enums are also different.

          Show
          andrew.wang Andrew Wang added a comment - Based on my read, proto2 is the default parsing mode, we need to explicitly opt-in to proto3. This is nice, since we won't have to modify our existing proto files if we upgrade PB. I think the mode also only affects parsing of the .proto file. proto3 adds new types like maps and unions, which aren't available in proto2 mode. I think enums are also different.
          Hide
          ozawa Tsuyoshi Ozawa added a comment -

          I would like to try proto v2 mode on protoc 3.2 by using syntax = "proto2": maybe this is what Stack mentioned as compatibility mode. Let me know if I'm wrong.

          Show
          ozawa Tsuyoshi Ozawa added a comment - I would like to try proto v2 mode on protoc 3.2 by using syntax = "proto2": maybe this is what Stack mentioned as compatibility mode. Let me know if I'm wrong.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          protoc 2.4 is actually becoming hard to find: homebrew doesn't serve it up any more, making setting up new dev systems hard.

          At the same time, not only does proto3 appear to have wire changes. I have no faith whatsoever in point releases of protobuf being compatible with previous protoc-generated coded code. To be fair, Avro has a similar policy. They just appear to recognise that the "just rebuild everything" strategy doesn't work for us. That is: the ASF big data stack is distributed development system, whereas the google build is centralised.

          If PB can be shaded, then moving up would be good, and, with the shading, let us update at our own rate.

          Show
          stevel@apache.org Steve Loughran added a comment - protoc 2.4 is actually becoming hard to find: homebrew doesn't serve it up any more, making setting up new dev systems hard. At the same time, not only does proto3 appear to have wire changes. I have no faith whatsoever in point releases of protobuf being compatible with previous protoc-generated coded code . To be fair, Avro has a similar policy. They just appear to recognise that the "just rebuild everything" strategy doesn't work for us. That is: the ASF big data stack is distributed development system, whereas the google build is centralised. If PB can be shaded, then moving up would be good, and, with the shading, let us update at our own rate.
          Hide
          ozawa Tsuyoshi Ozawa added a comment - - edited

          Andrew, I've started discussion on ML: http://mail-archives.apache.org/mod_mbox/hadoop-yarn-dev/201703.mbox/%3CCAAD07OLiwnCH2%3D1gw79vz9nUWX-VOMy8KtGt2pj8_Ln32oeDnw%40mail.gmail.com%3E
          I summarized opinions by developers discussed on HADOOP-13363 and HDFS-11010. I'm waiting for the opinions by other developers.

          Thanks Stack and Allen for sharing the information. The experience and information you shared makes us get motivated to update protobuf's version, I think.

          Show
          ozawa Tsuyoshi Ozawa added a comment - - edited Andrew, I've started discussion on ML: http://mail-archives.apache.org/mod_mbox/hadoop-yarn-dev/201703.mbox/%3CCAAD07OLiwnCH2%3D1gw79vz9nUWX-VOMy8KtGt2pj8_Ln32oeDnw%40mail.gmail.com%3E I summarized opinions by developers discussed on HADOOP-13363 and HDFS-11010 . I'm waiting for the opinions by other developers. Thanks Stack and Allen for sharing the information. The experience and information you shared makes us get motivated to update protobuf's version, I think.
          Hide
          stack stack added a comment -

          HBase uses pb3 internally because you can avoid having to copy all bytes making a Message ("zero-copy serialization") at least on output (working on same for input) and because it has support for ByteBuffers. pb3 in pb2 mode – the default – is wire compatible in our tests.

          Show
          stack stack added a comment - HBase uses pb3 internally because you can avoid having to copy all bytes making a Message ("zero-copy serialization") at least on output (working on same for input) and because it has support for ByteBuffers. pb3 in pb2 mode – the default – is wire compatible in our tests.
          Hide
          aw Allen Wittenauer added a comment -

          I see two big gains going to at least a newer version:

          • protoc 2.5.0 doesn't work out of the box on a variety of chipsets and platforms (It was an issue raised on the mailing list a while back, so it is impacting users.)
          • Ease of contribution: protobuf 2.5.0 is no longer the default protoc that is getting installed on new OS deployments.

          The first point, IMO, is fairly important. The OOB compilation experience for non-x86, non-Linux is pretty awful and this would be a huge improvement.

          Show
          aw Allen Wittenauer added a comment - I see two big gains going to at least a newer version: protoc 2.5.0 doesn't work out of the box on a variety of chipsets and platforms (It was an issue raised on the mailing list a while back, so it is impacting users.) Ease of contribution: protobuf 2.5.0 is no longer the default protoc that is getting installed on new OS deployments. The first point, IMO, is fairly important. The OOB compilation experience for non-x86, non-Linux is pretty awful and this would be a huge improvement.
          Hide
          andrew.wang Andrew Wang added a comment -

          I see there's been motion on this JIRA, did we ever have the ML discussion? Is source compatibility handled by the shaded client? Did we test wire compatibility? Chris Douglas brought up some incompatibilities on HDFS-11010 that I don't think were resolved.

          I also asked this on HDFS-11010, but what do we gain from upgrading to a 3.x release of PB?

          Show
          andrew.wang Andrew Wang added a comment - I see there's been motion on this JIRA, did we ever have the ML discussion? Is source compatibility handled by the shaded client? Did we test wire compatibility? Chris Douglas brought up some incompatibilities on HDFS-11010 that I don't think were resolved. I also asked this on HDFS-11010 , but what do we gain from upgrading to a 3.x release of PB?
          Hide
          ozawa Tsuyoshi Ozawa added a comment -

          protoc fails to generate files? Let me survey for a while.

          Show
          ozawa Tsuyoshi Ozawa added a comment - protoc fails to generate files? Let me survey for a while.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 14m 22s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          0 mvndep 0m 15s Maven dependency ordering for branch
          -1 mvninstall 0m 27s root in trunk failed.
          -1 compile 0m 24s root in trunk failed.
          +1 checkstyle 1m 49s trunk passed
          -1 mvnsite 0m 10s hadoop-hdfs in trunk failed.
          -1 mvneclipse 0m 10s hadoop-hdfs in trunk failed.
          0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project
          -1 findbugs 0m 12s hadoop-hdfs in trunk failed.
          -1 javadoc 0m 13s hadoop-hdfs in trunk failed.
          0 mvndep 0m 19s Maven dependency ordering for patch
          +1 mvninstall 1m 5s the patch passed
          +1 compile 17m 35s the patch passed
          -1 javac 17m 35s root generated 1232 new + 0 unchanged - 0 fixed = 1232 total (was 0)
          +1 checkstyle 2m 5s root: The patch generated 0 new + 27 unchanged - 1 fixed = 27 total (was 28)
          +1 mvnsite 1m 37s the patch passed
          +1 mvneclipse 0m 58s the patch passed
          -1 shellcheck 0m 16s The patch generated 632 new + 104 unchanged - 0 fixed = 736 total (was 104)
          -0 shelldocs 0m 21s The patch generated 282 new + 104 unchanged - 0 fixed = 386 total (was 104)
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project
          +1 findbugs 2m 13s the patch passed
          -1 javadoc 1m 12s hadoop-hdfs-project_hadoop-hdfs generated 9 new + 0 unchanged - 0 fixed = 9 total (was 0)
          +1 unit 0m 29s hadoop-project in the patch passed.
          -1 unit 54m 5s hadoop-hdfs in the patch failed.
          -1 asflicense 0m 51s The patch generated 1 ASF License warnings.
          128m 8s



          Reason Tests
          Failed junit tests hadoop.hdfs.server.balancer.TestBalancerWithEncryptedTransfer
            hadoop.hdfs.TestHDFSFileSystemContract
            hadoop.hdfs.TestErasureCodingPolicyWithSnapshot
            hadoop.hdfs.TestEncryptedTransfer
            hadoop.hdfs.server.namenode.snapshot.TestSnapshotDiffReport
            hadoop.hdfs.server.namenode.TestQuotaWithStripedBlocks
            hadoop.hdfs.TestBlockStoragePolicy
            hadoop.hdfs.TestRollingUpgrade
            hadoop.hdfs.web.TestWebHDFSForHA
            hadoop.hdfs.server.balancer.TestBalancerWithMultipleNameNodes
            hadoop.hdfs.server.namenode.metrics.TestNameNodeMetrics
            hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints
            hadoop.hdfs.TestDFSRSDefault10x4StripedOutputStream
            hadoop.hdfs.server.blockmanagement.TestPendingReconstruction
            hadoop.hdfs.TestClientProtocolForPipelineRecovery
            hadoop.hdfs.server.namenode.TestFSEditLogLoader
            hadoop.fs.contract.hdfs.TestHDFSContractSeek
            hadoop.hdfs.TestPread
            hadoop.fs.viewfs.TestViewFsFileStatusHdfs
            hadoop.hdfs.TestWriteConfigurationToDFS
            hadoop.cli.TestAclCLIWithPosixAclInheritance
            hadoop.hdfs.TestDFSStripedOutputStreamWithFailure160
            hadoop.hdfs.TestParallelShortCircuitLegacyRead
            hadoop.hdfs.TestFileCreationClient
            hadoop.hdfs.server.datanode.TestDataNodeHotSwapVolumes
            hadoop.hdfs.TestLeaseRecoveryStriped
            hadoop.hdfs.server.diskbalancer.TestDiskBalancer
            hadoop.hdfs.server.datanode.fsdataset.impl.TestSpaceReservation
            hadoop.hdfs.server.namenode.TestUpgradeDomainBlockPlacementPolicy
            hadoop.hdfs.crypto.TestHdfsCryptoStreams
            hadoop.hdfs.server.datanode.TestTransferRbw
            hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewerWithStripedBlocks
            hadoop.hdfs.TestWriteBlockGetsBlockLengthHint
            hadoop.hdfs.server.namenode.TestAddBlock
            hadoop.hdfs.TestDatanodeDeath
            hadoop.fs.TestSymlinkHdfsDisable
            hadoop.hdfs.TestDFSStripedOutputStreamWithFailure150
            hadoop.hdfs.TestDFSInotifyEventInputStream
            hadoop.hdfs.TestDecommission
            hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewerForContentSummary
            hadoop.hdfs.server.namenode.TestCommitBlockWithInvalidGenStamp
            hadoop.hdfs.server.namenode.snapshot.TestSnapshotMetrics
            hadoop.hdfs.shortcircuit.TestShortCircuitLocalRead
            hadoop.hdfs.TestWriteReadStripedFile
            hadoop.hdfs.server.mover.TestMover
            hadoop.hdfs.TestDFSRemove
            hadoop.hdfs.TestAclsEndToEnd
            hadoop.fs.loadGenerator.TestLoadGenerator
            hadoop.hdfs.TestFileCorruption
            hadoop.hdfs.server.datanode.TestNNHandlesBlockReportPerStorage
            hadoop.hdfs.server.namenode.snapshot.TestSnapshotReplication
            hadoop.hdfs.server.datanode.TestDataNodeTcpNoDelay
            hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyWriter
            hadoop.hdfs.TestFileAppend4
            hadoop.hdfs.server.diskbalancer.TestDiskBalancerRPC
            hadoop.hdfs.server.namenode.ha.TestDNFencing
            hadoop.hdfs.TestDFSStripedOutputStreamWithFailure200
            hadoop.hdfs.TestDFSStripedOutputStreamWithFailure110
            hadoop.hdfs.server.namenode.snapshot.TestSnapshotFileLength
            hadoop.hdfs.server.datanode.TestNNHandlesCombinedBlockReport
            hadoop.security.TestPermission
            hadoop.hdfs.TestDFSShell
            hadoop.hdfs.server.blockmanagement.TestNodeCount
            hadoop.hdfs.server.namenode.TestDeleteRace
            hadoop.hdfs.server.namenode.ha.TestDNFencingWithReplication
            hadoop.hdfs.server.datanode.TestDataNodeRollingUpgrade
            hadoop.hdfs.web.TestWebHDFSXAttr
            hadoop.hdfs.TestDFSClientFailover
            hadoop.hdfs.TestModTime
            hadoop.hdfs.TestDistributedFileSystem
            hadoop.hdfs.web.TestWebHDFS
            hadoop.hdfs.server.namenode.snapshot.TestINodeFileUnderConstructionWithSnapshot
            hadoop.hdfs.server.blockmanagement.TestOverReplicatedBlocks
            hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistFiles
            hadoop.fs.TestWebHdfsFileContextMainOperations
            hadoop.hdfs.server.namenode.snapshot.TestFileContextSnapshot
            hadoop.hdfs.TestFileCreationDelete
            hadoop.hdfs.TestDFSUpgrade
            hadoop.hdfs.TestParallelRead
            hadoop.hdfs.TestTrashWithEncryptionZones
            hadoop.hdfs.TestDFSStripedOutputStreamWithFailure180
            hadoop.security.TestPermissionSymlinks
            hadoop.hdfs.TestDFSStripedOutputStreamWithFailure010
            hadoop.hdfs.server.blockmanagement.TestBlockManager
            hadoop.hdfs.server.datanode.TestTriggerBlockReport
            hadoop.hdfs.TestAbandonBlock
            hadoop.hdfs.server.mover.TestStorageMover
            hadoop.hdfs.server.datanode.TestDataNodeMetrics
            hadoop.hdfs.TestDatanodeLayoutUpgrade
            hadoop.hdfs.TestDataStream
            hadoop.hdfs.TestParallelShortCircuitReadUnCached
            hadoop.hdfs.server.namenode.snapshot.TestSnapshotDeletion
            hadoop.hdfs.TestWriteRead
            hadoop.hdfs.server.namenode.TestStripedINodeFile
            hadoop.hdfs.tools.TestDebugAdmin
            hadoop.hdfs.server.datanode.TestDeleteBlockPool
            hadoop.hdfs.TestParallelShortCircuitRead
            hadoop.hdfs.TestDFSXORStripedInputStream
            hadoop.hdfs.TestRenameWhileOpen
            hadoop.hdfs.TestInjectionForSimulatedStorage
            hadoop.hdfs.server.datanode.TestCachingStrategy
            hadoop.hdfs.server.namenode.TestDecommissioningStatus
            hadoop.hdfs.TestDFSStripedOutputStreamWithFailure080
            hadoop.hdfs.TestExternalBlockReader
            hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA
            hadoop.hdfs.TestHFlush
            hadoop.hdfs.server.namenode.TestDiskspaceQuotaUpdate
            hadoop.fs.contract.hdfs.TestHDFSContractOpen
            hadoop.hdfs.web.TestFSMainOperationsWebHdfs
            hadoop.hdfs.server.namenode.snapshot.TestSnapshotBlocksMap
            hadoop.fs.TestFcHdfsPermission
            hadoop.hdfs.TestDFSStripedOutputStreamWithFailure000
            hadoop.hdfs.server.blockmanagement.TestBlockStatsMXBean
            hadoop.hdfs.TestFileChecksum
            hadoop.hdfs.TestClose
            hadoop.hdfs.server.datanode.TestDirectoryScanner
            hadoop.hdfs.TestAppendSnapshotTruncate
            hadoop.hdfs.client.impl.TestBlockReaderFactory
            hadoop.hdfs.TestDFSRollback
            hadoop.hdfs.server.datanode.TestBlockScanner
            hadoop.hdfs.TestLeaseRecovery2
            hadoop.hdfs.TestSnapshotCommands
            hadoop.hdfs.server.datanode.TestBatchIbr
            hadoop.fs.viewfs.TestViewFileSystemWithTruncate
            hadoop.hdfs.server.namenode.TestNameNodeMetadataConsistency
            hadoop.hdfs.server.blockmanagement.TestRBWBlockInvalidation
            hadoop.hdfs.server.datanode.TestReadOnlySharedStorage
            hadoop.hdfs.TestDFSStripedOutputStreamWithFailure
            hadoop.hdfs.TestLeaseRecovery
            hadoop.hdfs.TestRestartDFS
            hadoop.fs.contract.hdfs.TestHDFSContractMkdir
            hadoop.hdfs.client.impl.TestBlockReaderLocalLegacy
            hadoop.hdfs.server.namenode.TestNamenodeRetryCache
            hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure
            hadoop.hdfs.TestDFSFinalize
            hadoop.hdfs.server.blockmanagement.TestNameNodePrunesMissingStorages
            hadoop.hdfs.TestMiniDFSCluster
            hadoop.hdfs.TestAppendDifferentChecksum
            hadoop.hdfs.server.datanode.TestDataNodeMXBean
            hadoop.hdfs.server.namenode.TestFsckWithMultipleNameNodes
            hadoop.hdfs.TestDFSStripedOutputStream
            hadoop.hdfs.security.token.block.TestBlockToken
            hadoop.hdfs.TestDFSStorageStateRecovery
            hadoop.hdfs.server.namenode.ha.TestStandbyBlockManagement
            hadoop.hdfs.server.blockmanagement.TestSequentialBlockId
            hadoop.fs.TestUrlStreamHandler
            hadoop.hdfs.TestReplaceDatanodeOnFailure
            hadoop.hdfs.server.namenode.TestNestedEncryptionZones
            hadoop.hdfs.server.namenode.TestHostsFiles
            hadoop.hdfs.TestDFSPermission
            hadoop.hdfs.server.datanode.TestDiskError
            hadoop.hdfs.TestReadStripedFileWithDecoding
            hadoop.fs.TestSymlinkHdfsFileSystem
            hadoop.hdfs.TestListFilesInDFS
            hadoop.hdfs.server.namenode.TestParallelImageWrite
            hadoop.metrics2.sink.TestRollingFileSystemSinkWithHdfs
            hadoop.hdfs.server.namenode.ha.TestFailoverWithBlockTokensEnabled
            hadoop.hdfs.TestDFSStripedOutputStreamWithFailure120
            hadoop.hdfs.server.namenode.TestAuditLogger
            hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot
            hadoop.hdfs.server.namenode.TestNamenodeCapacityReport
            hadoop.hdfs.server.namenode.TestMetaSave
            hadoop.hdfs.server.blockmanagement.TestBlocksWithNotEnoughRacks
            hadoop.fs.viewfs.TestViewFileSystemAtHdfsRoot
            hadoop.hdfs.TestDFSRename
            hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl
            hadoop.hdfs.TestParallelUnixDomainRead
            hadoop.hdfs.TestUnsetAndChangeDirectoryEcPolicy
            hadoop.fs.viewfs.TestViewFsAtHdfsRoot
            hadoop.fs.TestSWebHdfsFileContextMainOperations
            hadoop.hdfs.protocol.datatransfer.sasl.TestSaslDataTransfer
            hadoop.hdfs.TestDFSStripedOutputStreamWithFailure090
            hadoop.hdfs.server.datanode.TestFsDatasetCacheRevocation
            hadoop.hdfs.server.namenode.ha.TestHASafeMode
            hadoop.hdfs.server.diskbalancer.command.TestDiskBalancerCommand
            hadoop.hdfs.server.namenode.ha.TestStandbyIsHot
            hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistLockedMemory
            hadoop.hdfs.TestBlockMissingException
            hadoop.hdfs.tools.TestDFSAdmin
            hadoop.hdfs.client.impl.TestClientBlockVerification
            hadoop.hdfs.TestDataTransferProtocol
            hadoop.hdfs.TestDFSClientRetries
            hadoop.hdfs.TestQuota
            hadoop.hdfs.server.namenode.TestNameNodeMXBean
            hadoop.hdfs.server.namenode.snapshot.TestSnapshottableDirListing
            hadoop.hdfs.server.datanode.TestBlockHasMultipleReplicasOnSameDN
            hadoop.fs.contract.hdfs.TestHDFSContractDelete
            hadoop.hdfs.TestMissingBlocksAlert
            hadoop.hdfs.TestFSOutputSummer
            hadoop.hdfs.server.namenode.TestQuotaByStorageType
            hadoop.hdfs.server.namenode.TestAddStripedBlocks
            hadoop.hdfs.server.blockmanagement.TestPendingInvalidateBlock
            hadoop.hdfs.client.impl.TestBlockReaderLocal
            hadoop.hdfs.server.namenode.TestFileContextXAttr
            hadoop.hdfs.server.namenode.TestReconstructStripedBlocks
            hadoop.hdfs.server.namenode.TestFileTruncate
            hadoop.hdfs.server.namenode.TestAddStripedBlockInFBR
            hadoop.hdfs.TestFileConcurrentReader
            hadoop.hdfs.TestDFSStripedOutputStreamWithFailure020
            hadoop.hdfs.server.namenode.ha.TestPendingCorruptDnMessages
            hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFSStriped
            hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots
            hadoop.hdfs.server.namenode.ha.TestSeveralNameNodes
            hadoop.hdfs.TestFileAppend2
            hadoop.hdfs.server.namenode.TestNameNodeXAttr
            hadoop.fs.contract.hdfs.TestHDFSContractConcat
            hadoop.fs.TestEnhancedByteBufferAccess
            hadoop.hdfs.server.namenode.TestProcessCorruptBlocks
            hadoop.hdfs.web.TestWebHdfsFileSystemContract
            hadoop.hdfs.TestMultiThreadedHflush
            hadoop.hdfs.TestEncryptionZones
            hadoop.hdfs.security.TestDelegationToken
            hadoop.hdfs.TestConnCache
            hadoop.hdfs.server.namenode.TestFSImage
            hadoop.hdfs.server.namenode.TestHDFSConcat
            hadoop.hdfs.server.datanode.TestHSync
            hadoop.hdfs.TestRead
            hadoop.hdfs.server.namenode.TestSnapshotPathINodes
            hadoop.hdfs.TestSetTimes
            hadoop.hdfs.TestReadStripedFileWithMissingBlocks
            hadoop.hdfs.TestDisableConnCache
            hadoop.hdfs.server.namenode.snapshot.TestSnapshotRename
            hadoop.hdfs.TestDFSOutputStream
            hadoop.hdfs.TestSmallBlock
            hadoop.fs.TestFcHdfsSetUMask
            hadoop.hdfs.TestFsShellPermission
            hadoop.hdfs.TestEncryptionZonesWithHA
            hadoop.hdfs.TestDFSStripedOutputStreamWithFailure100
            hadoop.hdfs.server.namenode.TestBackupNode
            hadoop.tracing.TestTracing
            hadoop.hdfs.TestDFSRSDefault10x4StripedInputStream
            hadoop.fs.contract.hdfs.TestHDFSContractRename
            hadoop.hdfs.server.namenode.ha.TestHAStateTransitions
            hadoop.hdfs.TestGetFileChecksum
            hadoop.hdfs.TestApplyingStoragePolicy
            hadoop.hdfs.server.datanode.TestBlockRecovery
            hadoop.hdfs.TestFileAppend
            hadoop.hdfs.TestDFSClientExcludedNodes
            hadoop.hdfs.TestLocalDFS
            hadoop.hdfs.server.balancer.TestBalancer
            hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewerForAcl
            hadoop.hdfs.server.balancer.TestBalancerWithNodeGroup
            hadoop.hdfs.TestHdfsAdmin
            hadoop.hdfs.server.namenode.TestAddOverReplicatedStripedBlocks
            hadoop.hdfs.TestDFSStripedOutputStreamWithFailure140
            hadoop.hdfs.server.blockmanagement.TestComputeInvalidateWork
            hadoop.hdfs.server.namenode.snapshot.TestSnapshotNameWithInvalidCharacters
            hadoop.hdfs.TestDFSRSDefault10x4StripedOutputStreamWithFailure
            hadoop.fs.contract.hdfs.TestHDFSContractGetFileStatus
            hadoop.hdfs.server.datanode.TestDataNodeErasureCodingMetrics
            hadoop.hdfs.server.namenode.TestListCorruptFileBlocks
            hadoop.hdfs.TestDFSMkdirs
            hadoop.hdfs.TestEncryptionZonesWithKMS
            hadoop.fs.TestSymlinkHdfsFileContext
            hadoop.hdfs.TestDatanodeReport
            hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaRecovery
            hadoop.hdfs.TestDFSXORStripedOutputStream
            hadoop.hdfs.server.datanode.TestDataNodeFaultInjector
            hadoop.hdfs.TestFileStatus
            hadoop.hdfs.TestSafeMode
            hadoop.hdfs.server.datanode.TestIncrementalBrVariations
            hadoop.hdfs.TestDFSStripedOutputStreamWithFailure130
            hadoop.hdfs.TestHDFSTrash
            hadoop.hdfs.server.namenode.ha.TestQuotasWithHA
            hadoop.hdfs.server.namenode.TestFSImageWithSnapshot
            hadoop.hdfs.server.namenode.TestFsck
            hadoop.hdfs.TestLease
            hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewer
            hadoop.hdfs.TestLargeBlock
            hadoop.metrics2.sink.TestRollingFileSystemSinkWithSecureHdfs
            hadoop.hdfs.server.namenode.ha.TestHAAppend
            hadoop.hdfs.server.blockmanagement.TestSequentialBlockGroupId
            hadoop.hdfs.server.datanode.TestBlockReplacement
            hadoop.hdfs.TestClientReportBadBlock
            hadoop.hdfs.protocolPB.TestPBHelper
            hadoop.hdfs.TestFileCreation
            hadoop.tracing.TestTracingShortCircuitLocalRead
            hadoop.hdfs.server.namenode.snapshot.TestNestedSnapshots
            hadoop.hdfs.TestSafeModeWithStripedFile
            hadoop.hdfs.TestMaintenanceState
            hadoop.hdfs.TestSetrepDecreasing
            hadoop.hdfs.tools.TestStoragePolicyCommands
            hadoop.hdfs.server.namenode.TestBlockUnderConstruction
            hadoop.hdfs.security.TestDelegationTokenForProxyUser
            hadoop.hdfs.server.namenode.ha.TestHAMetrics
            hadoop.hdfs.server.namenode.ha.TestPipelinesFailover
            hadoop.hdfs.web.TestWebHdfsWithRestCsrfPreventionFilter
            hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting
            hadoop.fs.shell.TestHdfsTextCommand
            hadoop.hdfs.TestDatanodeStartupFixesLegacyStorageIDs
            hadoop.cli.TestAclCLI
            hadoop.hdfs.TestSecureEncryptionZoneWithKMS
            hadoop.hdfs.TestBlocksScheduledCounter
            hadoop.hdfs.server.namenode.TestFileLimit
            hadoop.hdfs.server.namenode.TestFSDirectory
            hadoop.hdfs.TestReservedRawPaths
            hadoop.tools.TestJMXGet
            hadoop.hdfs.TestDFSStripedOutputStreamWithFailure170
            hadoop.hdfs.TestDFSInputStream
            hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks
            hadoop.hdfs.tools.offlineEditsViewer.TestOfflineEditsViewer
            hadoop.hdfs.TestDFSStripedOutputStreamWithFailure060
            hadoop.hdfs.TestDFSStripedInputStream
            hadoop.fs.TestFcHdfsCreateMkdir
            hadoop.hdfs.client.impl.TestBlockReaderRemote
            hadoop.hdfs.TestFileAppendRestart
            hadoop.hdfs.TestDFSStripedOutputStreamWithFailure030
            hadoop.hdfs.server.balancer.TestBalancerWithSaslDataTransfer
            hadoop.cli.TestHDFSCLI
            hadoop.hdfs.server.namenode.TestAuditLogs
            hadoop.fs.viewfs.TestViewFileSystemHdfs
            hadoop.fs.TestHDFSFileContextMainOperations
            hadoop.hdfs.TestFileLengthOnClusterRestart
            hadoop.hdfs.server.namenode.web.resources.TestWebHdfsDataLocality
            hadoop.hdfs.TestListFilesInFileContext
            hadoop.hdfs.server.namenode.TestCheckpoint
            hadoop.hdfs.server.datanode.TestDnRespectsBlockReportSplitThreshold
            hadoop.hdfs.server.namenode.snapshot.TestUpdatePipelineWithSnapshots
            hadoop.hdfs.TestCrcCorruption
            hadoop.hdfs.TestDFSStripedOutputStreamWithFailure210
            hadoop.hdfs.TestErasureCodeBenchmarkThroughput
            hadoop.hdfs.server.datanode.TestFsDatasetCache
            hadoop.hdfs.server.datanode.fsdataset.impl.TestInterDatanodeProtocol
            hadoop.hdfs.TestDFSStripedOutputStreamWithFailure040
            hadoop.hdfs.TestReadWhileWriting
            hadoop.hdfs.TestFileAppend3
            hadoop.hdfs.server.namenode.TestNNThroughputBenchmark
            hadoop.hdfs.TestSeekBug
            hadoop.hdfs.TestReplication
            hadoop.hdfs.TestGetBlocks
            hadoop.hdfs.TestDFSStripedOutputStreamWithFailure050
            hadoop.hdfs.TestDFSStartupVersions
            hadoop.fs.contract.hdfs.TestHDFSContractCreate
            hadoop.hdfs.TestDFSUpgradeFromImage
            hadoop.hdfs.TestDecommissionWithStriped
            hadoop.hdfs.web.TestHttpsFileSystem
            hadoop.hdfs.server.namenode.TestStartup
            hadoop.hdfs.TestParallelShortCircuitReadNoChecksum
            hadoop.hdfs.server.namenode.TestCacheDirectives
            hadoop.hdfs.TestDFSXORStripedOutputStreamWithFailure
            hadoop.hdfs.server.namenode.TestFavoredNodesEndToEnd
            hadoop.hdfs.TestReconstructStripedFile
            hadoop.hdfs.TestPipelines
            hadoop.hdfs.server.namenode.snapshot.TestGetContentSummaryWithSnapshot
            hadoop.hdfs.server.namenode.TestINodeFile
            hadoop.hdfs.server.datanode.TestDataNodeVolumeMetrics
            hadoop.hdfs.TestSetrepIncreasing
            hadoop.hdfs.server.namenode.ha.TestHarFileSystemWithHA
            hadoop.hdfs.shortcircuit.TestShortCircuitCache
            hadoop.hdfs.server.namenode.metrics.TestNNMetricFilesInGetListingOps
            hadoop.hdfs.TestDFSStripedOutputStreamWithFailure190
            hadoop.hdfs.server.datanode.fsdataset.impl.TestScrLazyPersistFiles
            hadoop.fs.contract.hdfs.TestHDFSContractAppend
            hadoop.hdfs.web.TestWebHdfsWithMultipleNameNodes
            hadoop.hdfs.server.namenode.snapshot.TestSetQuotaWithSnapshot
            hadoop.hdfs.server.namenode.snapshot.TestSnapshot
            hadoop.hdfs.server.datanode.fsdataset.impl.TestDatanodeRestart
            hadoop.hdfs.server.namenode.TestLargeDirectoryDelete
            hadoop.hdfs.TestTrashWithSecureEncryptionZones
            hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFS
            hadoop.fs.TestUnbuffer
            hadoop.hdfs.TestFSInputChecker
            hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaPlacement
            hadoop.fs.permission.TestStickyBit
            hadoop.hdfs.server.blockmanagement.TestReconstructStripedBlocksWithRackAwareness
            hadoop.fs.viewfs.TestViewFsDefaultValue
            hadoop.hdfs.TestDFSStripedOutputStreamWithFailure070
            hadoop.fs.viewfs.TestViewFsHdfs
            hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureToleration
            hadoop.hdfs.server.namenode.TestNameEditsConfigs
            hadoop.hdfs.TestDataTransferKeepalive
            hadoop.hdfs.server.balancer.TestBalancerWithHANameNodes
          Timed out junit tests org.apache.hadoop.hdfs.TestPersistBlocks



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:a9ad5d6
          JIRA Issue HADOOP-13363
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12860660/HADOOP-13363.005.patch
          Optional Tests asflicense shellcheck shelldocs compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
          uname Linux c6a49f671bf7 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 46d37a6
          Default Java 1.8.0_121
          mvninstall https://builds.apache.org/job/PreCommit-HADOOP-Build/11939/artifact/patchprocess/branch-mvninstall-root.txt
          compile https://builds.apache.org/job/PreCommit-HADOOP-Build/11939/artifact/patchprocess/branch-compile-root.txt
          mvnsite https://builds.apache.org/job/PreCommit-HADOOP-Build/11939/artifact/patchprocess/branch-mvnsite-hadoop-hdfs-project_hadoop-hdfs.txt
          mvneclipse https://builds.apache.org/job/PreCommit-HADOOP-Build/11939/artifact/patchprocess/branch-mvneclipse-hadoop-hdfs-project_hadoop-hdfs.txt
          shellcheck v0.4.6
          findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/11939/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs.txt
          javadoc https://builds.apache.org/job/PreCommit-HADOOP-Build/11939/artifact/patchprocess/branch-javadoc-hadoop-hdfs-project_hadoop-hdfs.txt
          javac https://builds.apache.org/job/PreCommit-HADOOP-Build/11939/artifact/patchprocess/diff-compile-javac-root.txt
          shellcheck https://builds.apache.org/job/PreCommit-HADOOP-Build/11939/artifact/patchprocess/diff-patch-shellcheck.txt
          shelldocs https://builds.apache.org/job/PreCommit-HADOOP-Build/11939/artifact/patchprocess/diff-patch-shelldocs.txt
          javadoc https://builds.apache.org/job/PreCommit-HADOOP-Build/11939/artifact/patchprocess/diff-javadoc-javadoc-hadoop-hdfs-project_hadoop-hdfs.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/11939/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/11939/testReport/
          asflicense https://builds.apache.org/job/PreCommit-HADOOP-Build/11939/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-project hadoop-hdfs-project/hadoop-hdfs U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11939/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 14m 22s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. 0 mvndep 0m 15s Maven dependency ordering for branch -1 mvninstall 0m 27s root in trunk failed. -1 compile 0m 24s root in trunk failed. +1 checkstyle 1m 49s trunk passed -1 mvnsite 0m 10s hadoop-hdfs in trunk failed. -1 mvneclipse 0m 10s hadoop-hdfs in trunk failed. 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project -1 findbugs 0m 12s hadoop-hdfs in trunk failed. -1 javadoc 0m 13s hadoop-hdfs in trunk failed. 0 mvndep 0m 19s Maven dependency ordering for patch +1 mvninstall 1m 5s the patch passed +1 compile 17m 35s the patch passed -1 javac 17m 35s root generated 1232 new + 0 unchanged - 0 fixed = 1232 total (was 0) +1 checkstyle 2m 5s root: The patch generated 0 new + 27 unchanged - 1 fixed = 27 total (was 28) +1 mvnsite 1m 37s the patch passed +1 mvneclipse 0m 58s the patch passed -1 shellcheck 0m 16s The patch generated 632 new + 104 unchanged - 0 fixed = 736 total (was 104) -0 shelldocs 0m 21s The patch generated 282 new + 104 unchanged - 0 fixed = 386 total (was 104) +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project +1 findbugs 2m 13s the patch passed -1 javadoc 1m 12s hadoop-hdfs-project_hadoop-hdfs generated 9 new + 0 unchanged - 0 fixed = 9 total (was 0) +1 unit 0m 29s hadoop-project in the patch passed. -1 unit 54m 5s hadoop-hdfs in the patch failed. -1 asflicense 0m 51s The patch generated 1 ASF License warnings. 128m 8s Reason Tests Failed junit tests hadoop.hdfs.server.balancer.TestBalancerWithEncryptedTransfer   hadoop.hdfs.TestHDFSFileSystemContract   hadoop.hdfs.TestErasureCodingPolicyWithSnapshot   hadoop.hdfs.TestEncryptedTransfer   hadoop.hdfs.server.namenode.snapshot.TestSnapshotDiffReport   hadoop.hdfs.server.namenode.TestQuotaWithStripedBlocks   hadoop.hdfs.TestBlockStoragePolicy   hadoop.hdfs.TestRollingUpgrade   hadoop.hdfs.web.TestWebHDFSForHA   hadoop.hdfs.server.balancer.TestBalancerWithMultipleNameNodes   hadoop.hdfs.server.namenode.metrics.TestNameNodeMetrics   hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints   hadoop.hdfs.TestDFSRSDefault10x4StripedOutputStream   hadoop.hdfs.server.blockmanagement.TestPendingReconstruction   hadoop.hdfs.TestClientProtocolForPipelineRecovery   hadoop.hdfs.server.namenode.TestFSEditLogLoader   hadoop.fs.contract.hdfs.TestHDFSContractSeek   hadoop.hdfs.TestPread   hadoop.fs.viewfs.TestViewFsFileStatusHdfs   hadoop.hdfs.TestWriteConfigurationToDFS   hadoop.cli.TestAclCLIWithPosixAclInheritance   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure160   hadoop.hdfs.TestParallelShortCircuitLegacyRead   hadoop.hdfs.TestFileCreationClient   hadoop.hdfs.server.datanode.TestDataNodeHotSwapVolumes   hadoop.hdfs.TestLeaseRecoveryStriped   hadoop.hdfs.server.diskbalancer.TestDiskBalancer   hadoop.hdfs.server.datanode.fsdataset.impl.TestSpaceReservation   hadoop.hdfs.server.namenode.TestUpgradeDomainBlockPlacementPolicy   hadoop.hdfs.crypto.TestHdfsCryptoStreams   hadoop.hdfs.server.datanode.TestTransferRbw   hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewerWithStripedBlocks   hadoop.hdfs.TestWriteBlockGetsBlockLengthHint   hadoop.hdfs.server.namenode.TestAddBlock   hadoop.hdfs.TestDatanodeDeath   hadoop.fs.TestSymlinkHdfsDisable   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure150   hadoop.hdfs.TestDFSInotifyEventInputStream   hadoop.hdfs.TestDecommission   hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewerForContentSummary   hadoop.hdfs.server.namenode.TestCommitBlockWithInvalidGenStamp   hadoop.hdfs.server.namenode.snapshot.TestSnapshotMetrics   hadoop.hdfs.shortcircuit.TestShortCircuitLocalRead   hadoop.hdfs.TestWriteReadStripedFile   hadoop.hdfs.server.mover.TestMover   hadoop.hdfs.TestDFSRemove   hadoop.hdfs.TestAclsEndToEnd   hadoop.fs.loadGenerator.TestLoadGenerator   hadoop.hdfs.TestFileCorruption   hadoop.hdfs.server.datanode.TestNNHandlesBlockReportPerStorage   hadoop.hdfs.server.namenode.snapshot.TestSnapshotReplication   hadoop.hdfs.server.datanode.TestDataNodeTcpNoDelay   hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyWriter   hadoop.hdfs.TestFileAppend4   hadoop.hdfs.server.diskbalancer.TestDiskBalancerRPC   hadoop.hdfs.server.namenode.ha.TestDNFencing   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure200   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure110   hadoop.hdfs.server.namenode.snapshot.TestSnapshotFileLength   hadoop.hdfs.server.datanode.TestNNHandlesCombinedBlockReport   hadoop.security.TestPermission   hadoop.hdfs.TestDFSShell   hadoop.hdfs.server.blockmanagement.TestNodeCount   hadoop.hdfs.server.namenode.TestDeleteRace   hadoop.hdfs.server.namenode.ha.TestDNFencingWithReplication   hadoop.hdfs.server.datanode.TestDataNodeRollingUpgrade   hadoop.hdfs.web.TestWebHDFSXAttr   hadoop.hdfs.TestDFSClientFailover   hadoop.hdfs.TestModTime   hadoop.hdfs.TestDistributedFileSystem   hadoop.hdfs.web.TestWebHDFS   hadoop.hdfs.server.namenode.snapshot.TestINodeFileUnderConstructionWithSnapshot   hadoop.hdfs.server.blockmanagement.TestOverReplicatedBlocks   hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistFiles   hadoop.fs.TestWebHdfsFileContextMainOperations   hadoop.hdfs.server.namenode.snapshot.TestFileContextSnapshot   hadoop.hdfs.TestFileCreationDelete   hadoop.hdfs.TestDFSUpgrade   hadoop.hdfs.TestParallelRead   hadoop.hdfs.TestTrashWithEncryptionZones   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure180   hadoop.security.TestPermissionSymlinks   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure010   hadoop.hdfs.server.blockmanagement.TestBlockManager   hadoop.hdfs.server.datanode.TestTriggerBlockReport   hadoop.hdfs.TestAbandonBlock   hadoop.hdfs.server.mover.TestStorageMover   hadoop.hdfs.server.datanode.TestDataNodeMetrics   hadoop.hdfs.TestDatanodeLayoutUpgrade   hadoop.hdfs.TestDataStream   hadoop.hdfs.TestParallelShortCircuitReadUnCached   hadoop.hdfs.server.namenode.snapshot.TestSnapshotDeletion   hadoop.hdfs.TestWriteRead   hadoop.hdfs.server.namenode.TestStripedINodeFile   hadoop.hdfs.tools.TestDebugAdmin   hadoop.hdfs.server.datanode.TestDeleteBlockPool   hadoop.hdfs.TestParallelShortCircuitRead   hadoop.hdfs.TestDFSXORStripedInputStream   hadoop.hdfs.TestRenameWhileOpen   hadoop.hdfs.TestInjectionForSimulatedStorage   hadoop.hdfs.server.datanode.TestCachingStrategy   hadoop.hdfs.server.namenode.TestDecommissioningStatus   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure080   hadoop.hdfs.TestExternalBlockReader   hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA   hadoop.hdfs.TestHFlush   hadoop.hdfs.server.namenode.TestDiskspaceQuotaUpdate   hadoop.fs.contract.hdfs.TestHDFSContractOpen   hadoop.hdfs.web.TestFSMainOperationsWebHdfs   hadoop.hdfs.server.namenode.snapshot.TestSnapshotBlocksMap   hadoop.fs.TestFcHdfsPermission   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure000   hadoop.hdfs.server.blockmanagement.TestBlockStatsMXBean   hadoop.hdfs.TestFileChecksum   hadoop.hdfs.TestClose   hadoop.hdfs.server.datanode.TestDirectoryScanner   hadoop.hdfs.TestAppendSnapshotTruncate   hadoop.hdfs.client.impl.TestBlockReaderFactory   hadoop.hdfs.TestDFSRollback   hadoop.hdfs.server.datanode.TestBlockScanner   hadoop.hdfs.TestLeaseRecovery2   hadoop.hdfs.TestSnapshotCommands   hadoop.hdfs.server.datanode.TestBatchIbr   hadoop.fs.viewfs.TestViewFileSystemWithTruncate   hadoop.hdfs.server.namenode.TestNameNodeMetadataConsistency   hadoop.hdfs.server.blockmanagement.TestRBWBlockInvalidation   hadoop.hdfs.server.datanode.TestReadOnlySharedStorage   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure   hadoop.hdfs.TestLeaseRecovery   hadoop.hdfs.TestRestartDFS   hadoop.fs.contract.hdfs.TestHDFSContractMkdir   hadoop.hdfs.client.impl.TestBlockReaderLocalLegacy   hadoop.hdfs.server.namenode.TestNamenodeRetryCache   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure   hadoop.hdfs.TestDFSFinalize   hadoop.hdfs.server.blockmanagement.TestNameNodePrunesMissingStorages   hadoop.hdfs.TestMiniDFSCluster   hadoop.hdfs.TestAppendDifferentChecksum   hadoop.hdfs.server.datanode.TestDataNodeMXBean   hadoop.hdfs.server.namenode.TestFsckWithMultipleNameNodes   hadoop.hdfs.TestDFSStripedOutputStream   hadoop.hdfs.security.token.block.TestBlockToken   hadoop.hdfs.TestDFSStorageStateRecovery   hadoop.hdfs.server.namenode.ha.TestStandbyBlockManagement   hadoop.hdfs.server.blockmanagement.TestSequentialBlockId   hadoop.fs.TestUrlStreamHandler   hadoop.hdfs.TestReplaceDatanodeOnFailure   hadoop.hdfs.server.namenode.TestNestedEncryptionZones   hadoop.hdfs.server.namenode.TestHostsFiles   hadoop.hdfs.TestDFSPermission   hadoop.hdfs.server.datanode.TestDiskError   hadoop.hdfs.TestReadStripedFileWithDecoding   hadoop.fs.TestSymlinkHdfsFileSystem   hadoop.hdfs.TestListFilesInDFS   hadoop.hdfs.server.namenode.TestParallelImageWrite   hadoop.metrics2.sink.TestRollingFileSystemSinkWithHdfs   hadoop.hdfs.server.namenode.ha.TestFailoverWithBlockTokensEnabled   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure120   hadoop.hdfs.server.namenode.TestAuditLogger   hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot   hadoop.hdfs.server.namenode.TestNamenodeCapacityReport   hadoop.hdfs.server.namenode.TestMetaSave   hadoop.hdfs.server.blockmanagement.TestBlocksWithNotEnoughRacks   hadoop.fs.viewfs.TestViewFileSystemAtHdfsRoot   hadoop.hdfs.TestDFSRename   hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl   hadoop.hdfs.TestParallelUnixDomainRead   hadoop.hdfs.TestUnsetAndChangeDirectoryEcPolicy   hadoop.fs.viewfs.TestViewFsAtHdfsRoot   hadoop.fs.TestSWebHdfsFileContextMainOperations   hadoop.hdfs.protocol.datatransfer.sasl.TestSaslDataTransfer   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure090   hadoop.hdfs.server.datanode.TestFsDatasetCacheRevocation   hadoop.hdfs.server.namenode.ha.TestHASafeMode   hadoop.hdfs.server.diskbalancer.command.TestDiskBalancerCommand   hadoop.hdfs.server.namenode.ha.TestStandbyIsHot   hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistLockedMemory   hadoop.hdfs.TestBlockMissingException   hadoop.hdfs.tools.TestDFSAdmin   hadoop.hdfs.client.impl.TestClientBlockVerification   hadoop.hdfs.TestDataTransferProtocol   hadoop.hdfs.TestDFSClientRetries   hadoop.hdfs.TestQuota   hadoop.hdfs.server.namenode.TestNameNodeMXBean   hadoop.hdfs.server.namenode.snapshot.TestSnapshottableDirListing   hadoop.hdfs.server.datanode.TestBlockHasMultipleReplicasOnSameDN   hadoop.fs.contract.hdfs.TestHDFSContractDelete   hadoop.hdfs.TestMissingBlocksAlert   hadoop.hdfs.TestFSOutputSummer   hadoop.hdfs.server.namenode.TestQuotaByStorageType   hadoop.hdfs.server.namenode.TestAddStripedBlocks   hadoop.hdfs.server.blockmanagement.TestPendingInvalidateBlock   hadoop.hdfs.client.impl.TestBlockReaderLocal   hadoop.hdfs.server.namenode.TestFileContextXAttr   hadoop.hdfs.server.namenode.TestReconstructStripedBlocks   hadoop.hdfs.server.namenode.TestFileTruncate   hadoop.hdfs.server.namenode.TestAddStripedBlockInFBR   hadoop.hdfs.TestFileConcurrentReader   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure020   hadoop.hdfs.server.namenode.ha.TestPendingCorruptDnMessages   hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFSStriped   hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots   hadoop.hdfs.server.namenode.ha.TestSeveralNameNodes   hadoop.hdfs.TestFileAppend2   hadoop.hdfs.server.namenode.TestNameNodeXAttr   hadoop.fs.contract.hdfs.TestHDFSContractConcat   hadoop.fs.TestEnhancedByteBufferAccess   hadoop.hdfs.server.namenode.TestProcessCorruptBlocks   hadoop.hdfs.web.TestWebHdfsFileSystemContract   hadoop.hdfs.TestMultiThreadedHflush   hadoop.hdfs.TestEncryptionZones   hadoop.hdfs.security.TestDelegationToken   hadoop.hdfs.TestConnCache   hadoop.hdfs.server.namenode.TestFSImage   hadoop.hdfs.server.namenode.TestHDFSConcat   hadoop.hdfs.server.datanode.TestHSync   hadoop.hdfs.TestRead   hadoop.hdfs.server.namenode.TestSnapshotPathINodes   hadoop.hdfs.TestSetTimes   hadoop.hdfs.TestReadStripedFileWithMissingBlocks   hadoop.hdfs.TestDisableConnCache   hadoop.hdfs.server.namenode.snapshot.TestSnapshotRename   hadoop.hdfs.TestDFSOutputStream   hadoop.hdfs.TestSmallBlock   hadoop.fs.TestFcHdfsSetUMask   hadoop.hdfs.TestFsShellPermission   hadoop.hdfs.TestEncryptionZonesWithHA   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure100   hadoop.hdfs.server.namenode.TestBackupNode   hadoop.tracing.TestTracing   hadoop.hdfs.TestDFSRSDefault10x4StripedInputStream   hadoop.fs.contract.hdfs.TestHDFSContractRename   hadoop.hdfs.server.namenode.ha.TestHAStateTransitions   hadoop.hdfs.TestGetFileChecksum   hadoop.hdfs.TestApplyingStoragePolicy   hadoop.hdfs.server.datanode.TestBlockRecovery   hadoop.hdfs.TestFileAppend   hadoop.hdfs.TestDFSClientExcludedNodes   hadoop.hdfs.TestLocalDFS   hadoop.hdfs.server.balancer.TestBalancer   hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewerForAcl   hadoop.hdfs.server.balancer.TestBalancerWithNodeGroup   hadoop.hdfs.TestHdfsAdmin   hadoop.hdfs.server.namenode.TestAddOverReplicatedStripedBlocks   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure140   hadoop.hdfs.server.blockmanagement.TestComputeInvalidateWork   hadoop.hdfs.server.namenode.snapshot.TestSnapshotNameWithInvalidCharacters   hadoop.hdfs.TestDFSRSDefault10x4StripedOutputStreamWithFailure   hadoop.fs.contract.hdfs.TestHDFSContractGetFileStatus   hadoop.hdfs.server.datanode.TestDataNodeErasureCodingMetrics   hadoop.hdfs.server.namenode.TestListCorruptFileBlocks   hadoop.hdfs.TestDFSMkdirs   hadoop.hdfs.TestEncryptionZonesWithKMS   hadoop.fs.TestSymlinkHdfsFileContext   hadoop.hdfs.TestDatanodeReport   hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaRecovery   hadoop.hdfs.TestDFSXORStripedOutputStream   hadoop.hdfs.server.datanode.TestDataNodeFaultInjector   hadoop.hdfs.TestFileStatus   hadoop.hdfs.TestSafeMode   hadoop.hdfs.server.datanode.TestIncrementalBrVariations   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure130   hadoop.hdfs.TestHDFSTrash   hadoop.hdfs.server.namenode.ha.TestQuotasWithHA   hadoop.hdfs.server.namenode.TestFSImageWithSnapshot   hadoop.hdfs.server.namenode.TestFsck   hadoop.hdfs.TestLease   hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewer   hadoop.hdfs.TestLargeBlock   hadoop.metrics2.sink.TestRollingFileSystemSinkWithSecureHdfs   hadoop.hdfs.server.namenode.ha.TestHAAppend   hadoop.hdfs.server.blockmanagement.TestSequentialBlockGroupId   hadoop.hdfs.server.datanode.TestBlockReplacement   hadoop.hdfs.TestClientReportBadBlock   hadoop.hdfs.protocolPB.TestPBHelper   hadoop.hdfs.TestFileCreation   hadoop.tracing.TestTracingShortCircuitLocalRead   hadoop.hdfs.server.namenode.snapshot.TestNestedSnapshots   hadoop.hdfs.TestSafeModeWithStripedFile   hadoop.hdfs.TestMaintenanceState   hadoop.hdfs.TestSetrepDecreasing   hadoop.hdfs.tools.TestStoragePolicyCommands   hadoop.hdfs.server.namenode.TestBlockUnderConstruction   hadoop.hdfs.security.TestDelegationTokenForProxyUser   hadoop.hdfs.server.namenode.ha.TestHAMetrics   hadoop.hdfs.server.namenode.ha.TestPipelinesFailover   hadoop.hdfs.web.TestWebHdfsWithRestCsrfPreventionFilter   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting   hadoop.fs.shell.TestHdfsTextCommand   hadoop.hdfs.TestDatanodeStartupFixesLegacyStorageIDs   hadoop.cli.TestAclCLI   hadoop.hdfs.TestSecureEncryptionZoneWithKMS   hadoop.hdfs.TestBlocksScheduledCounter   hadoop.hdfs.server.namenode.TestFileLimit   hadoop.hdfs.server.namenode.TestFSDirectory   hadoop.hdfs.TestReservedRawPaths   hadoop.tools.TestJMXGet   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure170   hadoop.hdfs.TestDFSInputStream   hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks   hadoop.hdfs.tools.offlineEditsViewer.TestOfflineEditsViewer   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure060   hadoop.hdfs.TestDFSStripedInputStream   hadoop.fs.TestFcHdfsCreateMkdir   hadoop.hdfs.client.impl.TestBlockReaderRemote   hadoop.hdfs.TestFileAppendRestart   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure030   hadoop.hdfs.server.balancer.TestBalancerWithSaslDataTransfer   hadoop.cli.TestHDFSCLI   hadoop.hdfs.server.namenode.TestAuditLogs   hadoop.fs.viewfs.TestViewFileSystemHdfs   hadoop.fs.TestHDFSFileContextMainOperations   hadoop.hdfs.TestFileLengthOnClusterRestart   hadoop.hdfs.server.namenode.web.resources.TestWebHdfsDataLocality   hadoop.hdfs.TestListFilesInFileContext   hadoop.hdfs.server.namenode.TestCheckpoint   hadoop.hdfs.server.datanode.TestDnRespectsBlockReportSplitThreshold   hadoop.hdfs.server.namenode.snapshot.TestUpdatePipelineWithSnapshots   hadoop.hdfs.TestCrcCorruption   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure210   hadoop.hdfs.TestErasureCodeBenchmarkThroughput   hadoop.hdfs.server.datanode.TestFsDatasetCache   hadoop.hdfs.server.datanode.fsdataset.impl.TestInterDatanodeProtocol   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure040   hadoop.hdfs.TestReadWhileWriting   hadoop.hdfs.TestFileAppend3   hadoop.hdfs.server.namenode.TestNNThroughputBenchmark   hadoop.hdfs.TestSeekBug   hadoop.hdfs.TestReplication   hadoop.hdfs.TestGetBlocks   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure050   hadoop.hdfs.TestDFSStartupVersions   hadoop.fs.contract.hdfs.TestHDFSContractCreate   hadoop.hdfs.TestDFSUpgradeFromImage   hadoop.hdfs.TestDecommissionWithStriped   hadoop.hdfs.web.TestHttpsFileSystem   hadoop.hdfs.server.namenode.TestStartup   hadoop.hdfs.TestParallelShortCircuitReadNoChecksum   hadoop.hdfs.server.namenode.TestCacheDirectives   hadoop.hdfs.TestDFSXORStripedOutputStreamWithFailure   hadoop.hdfs.server.namenode.TestFavoredNodesEndToEnd   hadoop.hdfs.TestReconstructStripedFile   hadoop.hdfs.TestPipelines   hadoop.hdfs.server.namenode.snapshot.TestGetContentSummaryWithSnapshot   hadoop.hdfs.server.namenode.TestINodeFile   hadoop.hdfs.server.datanode.TestDataNodeVolumeMetrics   hadoop.hdfs.TestSetrepIncreasing   hadoop.hdfs.server.namenode.ha.TestHarFileSystemWithHA   hadoop.hdfs.shortcircuit.TestShortCircuitCache   hadoop.hdfs.server.namenode.metrics.TestNNMetricFilesInGetListingOps   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure190   hadoop.hdfs.server.datanode.fsdataset.impl.TestScrLazyPersistFiles   hadoop.fs.contract.hdfs.TestHDFSContractAppend   hadoop.hdfs.web.TestWebHdfsWithMultipleNameNodes   hadoop.hdfs.server.namenode.snapshot.TestSetQuotaWithSnapshot   hadoop.hdfs.server.namenode.snapshot.TestSnapshot   hadoop.hdfs.server.datanode.fsdataset.impl.TestDatanodeRestart   hadoop.hdfs.server.namenode.TestLargeDirectoryDelete   hadoop.hdfs.TestTrashWithSecureEncryptionZones   hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFS   hadoop.fs.TestUnbuffer   hadoop.hdfs.TestFSInputChecker   hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaPlacement   hadoop.fs.permission.TestStickyBit   hadoop.hdfs.server.blockmanagement.TestReconstructStripedBlocksWithRackAwareness   hadoop.fs.viewfs.TestViewFsDefaultValue   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure070   hadoop.fs.viewfs.TestViewFsHdfs   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureToleration   hadoop.hdfs.server.namenode.TestNameEditsConfigs   hadoop.hdfs.TestDataTransferKeepalive   hadoop.hdfs.server.balancer.TestBalancerWithHANameNodes Timed out junit tests org.apache.hadoop.hdfs.TestPersistBlocks Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HADOOP-13363 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12860660/HADOOP-13363.005.patch Optional Tests asflicense shellcheck shelldocs compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux c6a49f671bf7 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 46d37a6 Default Java 1.8.0_121 mvninstall https://builds.apache.org/job/PreCommit-HADOOP-Build/11939/artifact/patchprocess/branch-mvninstall-root.txt compile https://builds.apache.org/job/PreCommit-HADOOP-Build/11939/artifact/patchprocess/branch-compile-root.txt mvnsite https://builds.apache.org/job/PreCommit-HADOOP-Build/11939/artifact/patchprocess/branch-mvnsite-hadoop-hdfs-project_hadoop-hdfs.txt mvneclipse https://builds.apache.org/job/PreCommit-HADOOP-Build/11939/artifact/patchprocess/branch-mvneclipse-hadoop-hdfs-project_hadoop-hdfs.txt shellcheck v0.4.6 findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/11939/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs.txt javadoc https://builds.apache.org/job/PreCommit-HADOOP-Build/11939/artifact/patchprocess/branch-javadoc-hadoop-hdfs-project_hadoop-hdfs.txt javac https://builds.apache.org/job/PreCommit-HADOOP-Build/11939/artifact/patchprocess/diff-compile-javac-root.txt shellcheck https://builds.apache.org/job/PreCommit-HADOOP-Build/11939/artifact/patchprocess/diff-patch-shellcheck.txt shelldocs https://builds.apache.org/job/PreCommit-HADOOP-Build/11939/artifact/patchprocess/diff-patch-shelldocs.txt javadoc https://builds.apache.org/job/PreCommit-HADOOP-Build/11939/artifact/patchprocess/diff-javadoc-javadoc-hadoop-hdfs-project_hadoop-hdfs.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/11939/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/11939/testReport/ asflicense https://builds.apache.org/job/PreCommit-HADOOP-Build/11939/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-project hadoop-hdfs-project/hadoop-hdfs U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11939/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -

          A patch to the testing environment has been detected.
          Re-executing against the patched versions to perform further tests.
          The console is at https://builds.apache.org/job/PreCommit-HADOOP-Build/11939/console in case of problems.

          Show
          hadoopqa Hadoop QA added a comment - A patch to the testing environment has been detected. Re-executing against the patched versions to perform further tests. The console is at https://builds.apache.org/job/PreCommit-HADOOP-Build/11939/console in case of problems.
          Hide
          ozawa Tsuyoshi Ozawa added a comment -

          Trying to fix a problem by lack of execution permission.

          Show
          ozawa Tsuyoshi Ozawa added a comment - Trying to fix a problem by lack of execution permission.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 16m 41s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          0 mvndep 0m 17s Maven dependency ordering for branch
          -1 mvninstall 0m 32s root in trunk failed.
          -1 compile 0m 28s root in trunk failed.
          +1 checkstyle 1m 56s trunk passed
          -1 mvnsite 0m 11s hadoop-hdfs in trunk failed.
          -1 mvneclipse 0m 9s hadoop-hdfs in trunk failed.
          0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project
          -1 findbugs 0m 11s hadoop-hdfs in trunk failed.
          -1 javadoc 0m 11s hadoop-hdfs in trunk failed.
          0 mvndep 0m 22s Maven dependency ordering for patch
          -1 mvninstall 0m 12s hadoop-hdfs in the patch failed.
          -1 compile 0m 28s root in the patch failed.
          -1 javac 0m 28s root in the patch failed.
          +1 checkstyle 1m 57s root: The patch generated 0 new + 27 unchanged - 1 fixed = 27 total (was 28)
          -1 mvnsite 0m 15s hadoop-hdfs in the patch failed.
          -1 mvneclipse 0m 15s hadoop-hdfs in the patch failed.
          +1 shellcheck 0m 10s There were no new shellcheck issues.
          +1 shelldocs 0m 9s There were no new shelldocs issues.
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 2s The patch has no ill-formed XML file.
          0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project
          -1 findbugs 0m 16s hadoop-hdfs in the patch failed.
          -1 javadoc 0m 17s hadoop-hdfs in the patch failed.
          +1 unit 0m 16s hadoop-project in the patch passed.
          -1 unit 0m 18s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 27s The patch does not generate ASF License warnings.
          51m 29s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:a9ad5d6
          JIRA Issue HADOOP-13363
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12860647/HADOOP-13363.004.patch
          Optional Tests asflicense shellcheck shelldocs compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
          uname Linux da763511ca6f 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 96e2ab8
          Default Java 1.8.0_121
          mvninstall https://builds.apache.org/job/PreCommit-HADOOP-Build/11938/artifact/patchprocess/branch-mvninstall-root.txt
          compile https://builds.apache.org/job/PreCommit-HADOOP-Build/11938/artifact/patchprocess/branch-compile-root.txt
          mvnsite https://builds.apache.org/job/PreCommit-HADOOP-Build/11938/artifact/patchprocess/branch-mvnsite-hadoop-hdfs-project_hadoop-hdfs.txt
          mvneclipse https://builds.apache.org/job/PreCommit-HADOOP-Build/11938/artifact/patchprocess/branch-mvneclipse-hadoop-hdfs-project_hadoop-hdfs.txt
          shellcheck v0.4.6
          findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/11938/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs.txt
          javadoc https://builds.apache.org/job/PreCommit-HADOOP-Build/11938/artifact/patchprocess/branch-javadoc-hadoop-hdfs-project_hadoop-hdfs.txt
          mvninstall https://builds.apache.org/job/PreCommit-HADOOP-Build/11938/artifact/patchprocess/patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs.txt
          compile https://builds.apache.org/job/PreCommit-HADOOP-Build/11938/artifact/patchprocess/patch-compile-root.txt
          javac https://builds.apache.org/job/PreCommit-HADOOP-Build/11938/artifact/patchprocess/patch-compile-root.txt
          mvnsite https://builds.apache.org/job/PreCommit-HADOOP-Build/11938/artifact/patchprocess/patch-mvnsite-hadoop-hdfs-project_hadoop-hdfs.txt
          mvneclipse https://builds.apache.org/job/PreCommit-HADOOP-Build/11938/artifact/patchprocess/patch-mvneclipse-hadoop-hdfs-project_hadoop-hdfs.txt
          findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/11938/artifact/patchprocess/patch-findbugs-hadoop-hdfs-project_hadoop-hdfs.txt
          javadoc https://builds.apache.org/job/PreCommit-HADOOP-Build/11938/artifact/patchprocess/patch-javadoc-hadoop-hdfs-project_hadoop-hdfs.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/11938/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/11938/testReport/
          modules C: hadoop-project hadoop-hdfs-project/hadoop-hdfs U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11938/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 16m 41s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. 0 mvndep 0m 17s Maven dependency ordering for branch -1 mvninstall 0m 32s root in trunk failed. -1 compile 0m 28s root in trunk failed. +1 checkstyle 1m 56s trunk passed -1 mvnsite 0m 11s hadoop-hdfs in trunk failed. -1 mvneclipse 0m 9s hadoop-hdfs in trunk failed. 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project -1 findbugs 0m 11s hadoop-hdfs in trunk failed. -1 javadoc 0m 11s hadoop-hdfs in trunk failed. 0 mvndep 0m 22s Maven dependency ordering for patch -1 mvninstall 0m 12s hadoop-hdfs in the patch failed. -1 compile 0m 28s root in the patch failed. -1 javac 0m 28s root in the patch failed. +1 checkstyle 1m 57s root: The patch generated 0 new + 27 unchanged - 1 fixed = 27 total (was 28) -1 mvnsite 0m 15s hadoop-hdfs in the patch failed. -1 mvneclipse 0m 15s hadoop-hdfs in the patch failed. +1 shellcheck 0m 10s There were no new shellcheck issues. +1 shelldocs 0m 9s There were no new shelldocs issues. +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 2s The patch has no ill-formed XML file. 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project -1 findbugs 0m 16s hadoop-hdfs in the patch failed. -1 javadoc 0m 17s hadoop-hdfs in the patch failed. +1 unit 0m 16s hadoop-project in the patch passed. -1 unit 0m 18s hadoop-hdfs in the patch failed. +1 asflicense 0m 27s The patch does not generate ASF License warnings. 51m 29s Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HADOOP-13363 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12860647/HADOOP-13363.004.patch Optional Tests asflicense shellcheck shelldocs compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux da763511ca6f 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 96e2ab8 Default Java 1.8.0_121 mvninstall https://builds.apache.org/job/PreCommit-HADOOP-Build/11938/artifact/patchprocess/branch-mvninstall-root.txt compile https://builds.apache.org/job/PreCommit-HADOOP-Build/11938/artifact/patchprocess/branch-compile-root.txt mvnsite https://builds.apache.org/job/PreCommit-HADOOP-Build/11938/artifact/patchprocess/branch-mvnsite-hadoop-hdfs-project_hadoop-hdfs.txt mvneclipse https://builds.apache.org/job/PreCommit-HADOOP-Build/11938/artifact/patchprocess/branch-mvneclipse-hadoop-hdfs-project_hadoop-hdfs.txt shellcheck v0.4.6 findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/11938/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs.txt javadoc https://builds.apache.org/job/PreCommit-HADOOP-Build/11938/artifact/patchprocess/branch-javadoc-hadoop-hdfs-project_hadoop-hdfs.txt mvninstall https://builds.apache.org/job/PreCommit-HADOOP-Build/11938/artifact/patchprocess/patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs.txt compile https://builds.apache.org/job/PreCommit-HADOOP-Build/11938/artifact/patchprocess/patch-compile-root.txt javac https://builds.apache.org/job/PreCommit-HADOOP-Build/11938/artifact/patchprocess/patch-compile-root.txt mvnsite https://builds.apache.org/job/PreCommit-HADOOP-Build/11938/artifact/patchprocess/patch-mvnsite-hadoop-hdfs-project_hadoop-hdfs.txt mvneclipse https://builds.apache.org/job/PreCommit-HADOOP-Build/11938/artifact/patchprocess/patch-mvneclipse-hadoop-hdfs-project_hadoop-hdfs.txt findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/11938/artifact/patchprocess/patch-findbugs-hadoop-hdfs-project_hadoop-hdfs.txt javadoc https://builds.apache.org/job/PreCommit-HADOOP-Build/11938/artifact/patchprocess/patch-javadoc-hadoop-hdfs-project_hadoop-hdfs.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/11938/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/11938/testReport/ modules C: hadoop-project hadoop-hdfs-project/hadoop-hdfs U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11938/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -

          A patch to the testing environment has been detected.
          Re-executing against the patched versions to perform further tests.
          The console is at https://builds.apache.org/job/PreCommit-HADOOP-Build/11938/console in case of problems.

          Show
          hadoopqa Hadoop QA added a comment - A patch to the testing environment has been detected. Re-executing against the patched versions to perform further tests. The console is at https://builds.apache.org/job/PreCommit-HADOOP-Build/11938/console in case of problems.
          Hide
          ozawa Tsuyoshi Ozawa added a comment -

          Thanks Allen! Updated.

          @stack unfortunately, not yet. I've also checked discussion on HDFS-11010. Based on the discussion on the jira and here, I would like to start next argument.

          Show
          ozawa Tsuyoshi Ozawa added a comment - Thanks Allen! Updated. @stack unfortunately, not yet. I've also checked discussion on HDFS-11010 . Based on the discussion on the jira and here, I would like to start next argument.
          Hide
          ozawa Tsuyoshi Ozawa added a comment -

          Attaching a patch to update Dockerfile to download protobuf 3.2.0.

          Show
          ozawa Tsuyoshi Ozawa added a comment - Attaching a patch to update Dockerfile to download protobuf 3.2.0.
          Hide
          aw Allen Wittenauer added a comment -

          (It might also be time to move to Xenial from Trusty. I think I've got a working Dockerfile around here that does that too.)

          Show
          aw Allen Wittenauer added a comment - (It might also be time to move to Xenial from Trusty. I think I've got a working Dockerfile around here that does that too.)
          Hide
          aw Allen Wittenauer added a comment -

          Failed to execute goal org.apache.hadoop:hadoop-maven-plugins:3.0.0-alpha3-SNAPSHOT:protoc (compile-protoc)

          i.e., dev-support/docker/Dockerfile needs to be updated to use the same version of protobuf that's in the maven pom.xml file.

          Show
          aw Allen Wittenauer added a comment - Failed to execute goal org.apache.hadoop:hadoop-maven-plugins:3.0.0-alpha3-SNAPSHOT:protoc (compile-protoc) i.e., dev-support/docker/Dockerfile needs to be updated to use the same version of protobuf that's in the maven pom.xml file.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 16s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          0 mvndep 0m 14s Maven dependency ordering for branch
          +1 mvninstall 12m 46s trunk passed
          +1 compile 16m 4s trunk passed
          +1 checkstyle 1m 54s trunk passed
          +1 mvnsite 1m 16s trunk passed
          +1 mvneclipse 0m 37s trunk passed
          0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project
          +1 findbugs 1m 53s trunk passed
          +1 javadoc 1m 5s trunk passed
          0 mvndep 0m 19s Maven dependency ordering for patch
          -1 mvninstall 0m 9s hadoop-hdfs in the patch failed.
          -1 compile 0m 25s root in the patch failed.
          -1 javac 0m 25s root in the patch failed.
          +1 checkstyle 1m 50s root: The patch generated 0 new + 27 unchanged - 1 fixed = 27 total (was 28)
          -1 mvnsite 0m 12s hadoop-hdfs in the patch failed.
          -1 mvneclipse 0m 11s hadoop-hdfs in the patch failed.
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project
          -1 findbugs 0m 12s hadoop-hdfs in the patch failed.
          -1 javadoc 0m 12s hadoop-hdfs in the patch failed.
          +1 unit 0m 13s hadoop-project in the patch passed.
          -1 unit 0m 12s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 22s The patch does not generate ASF License warnings.
          65m 23s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:a9ad5d6
          JIRA Issue HADOOP-13363
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12860517/HADOOP-13363.003.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
          uname Linux f0297af43055 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 84ddedc
          Default Java 1.8.0_121
          findbugs v3.0.0
          mvninstall https://builds.apache.org/job/PreCommit-HADOOP-Build/11932/artifact/patchprocess/patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs.txt
          compile https://builds.apache.org/job/PreCommit-HADOOP-Build/11932/artifact/patchprocess/patch-compile-root.txt
          javac https://builds.apache.org/job/PreCommit-HADOOP-Build/11932/artifact/patchprocess/patch-compile-root.txt
          mvnsite https://builds.apache.org/job/PreCommit-HADOOP-Build/11932/artifact/patchprocess/patch-mvnsite-hadoop-hdfs-project_hadoop-hdfs.txt
          mvneclipse https://builds.apache.org/job/PreCommit-HADOOP-Build/11932/artifact/patchprocess/patch-mvneclipse-hadoop-hdfs-project_hadoop-hdfs.txt
          findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/11932/artifact/patchprocess/patch-findbugs-hadoop-hdfs-project_hadoop-hdfs.txt
          javadoc https://builds.apache.org/job/PreCommit-HADOOP-Build/11932/artifact/patchprocess/patch-javadoc-hadoop-hdfs-project_hadoop-hdfs.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/11932/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/11932/testReport/
          modules C: hadoop-project hadoop-hdfs-project/hadoop-hdfs U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11932/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 16s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. 0 mvndep 0m 14s Maven dependency ordering for branch +1 mvninstall 12m 46s trunk passed +1 compile 16m 4s trunk passed +1 checkstyle 1m 54s trunk passed +1 mvnsite 1m 16s trunk passed +1 mvneclipse 0m 37s trunk passed 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project +1 findbugs 1m 53s trunk passed +1 javadoc 1m 5s trunk passed 0 mvndep 0m 19s Maven dependency ordering for patch -1 mvninstall 0m 9s hadoop-hdfs in the patch failed. -1 compile 0m 25s root in the patch failed. -1 javac 0m 25s root in the patch failed. +1 checkstyle 1m 50s root: The patch generated 0 new + 27 unchanged - 1 fixed = 27 total (was 28) -1 mvnsite 0m 12s hadoop-hdfs in the patch failed. -1 mvneclipse 0m 11s hadoop-hdfs in the patch failed. +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project -1 findbugs 0m 12s hadoop-hdfs in the patch failed. -1 javadoc 0m 12s hadoop-hdfs in the patch failed. +1 unit 0m 13s hadoop-project in the patch passed. -1 unit 0m 12s hadoop-hdfs in the patch failed. +1 asflicense 0m 22s The patch does not generate ASF License warnings. 65m 23s Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HADOOP-13363 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12860517/HADOOP-13363.003.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux f0297af43055 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 84ddedc Default Java 1.8.0_121 findbugs v3.0.0 mvninstall https://builds.apache.org/job/PreCommit-HADOOP-Build/11932/artifact/patchprocess/patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs.txt compile https://builds.apache.org/job/PreCommit-HADOOP-Build/11932/artifact/patchprocess/patch-compile-root.txt javac https://builds.apache.org/job/PreCommit-HADOOP-Build/11932/artifact/patchprocess/patch-compile-root.txt mvnsite https://builds.apache.org/job/PreCommit-HADOOP-Build/11932/artifact/patchprocess/patch-mvnsite-hadoop-hdfs-project_hadoop-hdfs.txt mvneclipse https://builds.apache.org/job/PreCommit-HADOOP-Build/11932/artifact/patchprocess/patch-mvneclipse-hadoop-hdfs-project_hadoop-hdfs.txt findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/11932/artifact/patchprocess/patch-findbugs-hadoop-hdfs-project_hadoop-hdfs.txt javadoc https://builds.apache.org/job/PreCommit-HADOOP-Build/11932/artifact/patchprocess/patch-javadoc-hadoop-hdfs-project_hadoop-hdfs.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/11932/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/11932/testReport/ modules C: hadoop-project hadoop-hdfs-project/hadoop-hdfs U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11932/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 22s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          0 mvndep 0m 16s Maven dependency ordering for branch
          +1 mvninstall 14m 29s trunk passed
          +1 compile 17m 30s trunk passed
          +1 checkstyle 1m 57s trunk passed
          +1 mvnsite 1m 14s trunk passed
          +1 mvneclipse 0m 36s trunk passed
          0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project
          +1 findbugs 1m 52s trunk passed
          +1 javadoc 1m 6s trunk passed
          0 mvndep 0m 19s Maven dependency ordering for patch
          -1 mvninstall 0m 9s hadoop-hdfs in the patch failed.
          -1 compile 0m 24s root in the patch failed.
          -1 javac 0m 24s root in the patch failed.
          +1 checkstyle 1m 48s root: The patch generated 0 new + 27 unchanged - 1 fixed = 27 total (was 28)
          -1 mvnsite 0m 12s hadoop-hdfs in the patch failed.
          -1 mvneclipse 0m 11s hadoop-hdfs in the patch failed.
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project
          -1 findbugs 0m 13s hadoop-hdfs in the patch failed.
          -1 javadoc 0m 13s hadoop-hdfs in the patch failed.
          +1 unit 0m 12s hadoop-project in the patch passed.
          -1 unit 0m 13s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 22s The patch does not generate ASF License warnings.
          68m 34s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:a9ad5d6
          JIRA Issue HADOOP-13363
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12860511/HADOOP-13363.002.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
          uname Linux 30dd0a8e3236 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 84ddedc
          Default Java 1.8.0_121
          findbugs v3.0.0
          mvninstall https://builds.apache.org/job/PreCommit-HADOOP-Build/11931/artifact/patchprocess/patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs.txt
          compile https://builds.apache.org/job/PreCommit-HADOOP-Build/11931/artifact/patchprocess/patch-compile-root.txt
          javac https://builds.apache.org/job/PreCommit-HADOOP-Build/11931/artifact/patchprocess/patch-compile-root.txt
          mvnsite https://builds.apache.org/job/PreCommit-HADOOP-Build/11931/artifact/patchprocess/patch-mvnsite-hadoop-hdfs-project_hadoop-hdfs.txt
          mvneclipse https://builds.apache.org/job/PreCommit-HADOOP-Build/11931/artifact/patchprocess/patch-mvneclipse-hadoop-hdfs-project_hadoop-hdfs.txt
          findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/11931/artifact/patchprocess/patch-findbugs-hadoop-hdfs-project_hadoop-hdfs.txt
          javadoc https://builds.apache.org/job/PreCommit-HADOOP-Build/11931/artifact/patchprocess/patch-javadoc-hadoop-hdfs-project_hadoop-hdfs.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/11931/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/11931/testReport/
          modules C: hadoop-project hadoop-hdfs-project/hadoop-hdfs U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11931/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 22s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. 0 mvndep 0m 16s Maven dependency ordering for branch +1 mvninstall 14m 29s trunk passed +1 compile 17m 30s trunk passed +1 checkstyle 1m 57s trunk passed +1 mvnsite 1m 14s trunk passed +1 mvneclipse 0m 36s trunk passed 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project +1 findbugs 1m 52s trunk passed +1 javadoc 1m 6s trunk passed 0 mvndep 0m 19s Maven dependency ordering for patch -1 mvninstall 0m 9s hadoop-hdfs in the patch failed. -1 compile 0m 24s root in the patch failed. -1 javac 0m 24s root in the patch failed. +1 checkstyle 1m 48s root: The patch generated 0 new + 27 unchanged - 1 fixed = 27 total (was 28) -1 mvnsite 0m 12s hadoop-hdfs in the patch failed. -1 mvneclipse 0m 11s hadoop-hdfs in the patch failed. +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project -1 findbugs 0m 13s hadoop-hdfs in the patch failed. -1 javadoc 0m 13s hadoop-hdfs in the patch failed. +1 unit 0m 12s hadoop-project in the patch passed. -1 unit 0m 13s hadoop-hdfs in the patch failed. +1 asflicense 0m 22s The patch does not generate ASF License warnings. 68m 34s Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HADOOP-13363 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12860511/HADOOP-13363.002.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux 30dd0a8e3236 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 84ddedc Default Java 1.8.0_121 findbugs v3.0.0 mvninstall https://builds.apache.org/job/PreCommit-HADOOP-Build/11931/artifact/patchprocess/patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs.txt compile https://builds.apache.org/job/PreCommit-HADOOP-Build/11931/artifact/patchprocess/patch-compile-root.txt javac https://builds.apache.org/job/PreCommit-HADOOP-Build/11931/artifact/patchprocess/patch-compile-root.txt mvnsite https://builds.apache.org/job/PreCommit-HADOOP-Build/11931/artifact/patchprocess/patch-mvnsite-hadoop-hdfs-project_hadoop-hdfs.txt mvneclipse https://builds.apache.org/job/PreCommit-HADOOP-Build/11931/artifact/patchprocess/patch-mvneclipse-hadoop-hdfs-project_hadoop-hdfs.txt findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/11931/artifact/patchprocess/patch-findbugs-hadoop-hdfs-project_hadoop-hdfs.txt javadoc https://builds.apache.org/job/PreCommit-HADOOP-Build/11931/artifact/patchprocess/patch-javadoc-hadoop-hdfs-project_hadoop-hdfs.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/11931/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/11931/testReport/ modules C: hadoop-project hadoop-hdfs-project/hadoop-hdfs U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11931/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 18s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 mvninstall 7m 7s trunk passed
          +1 compile 0m 8s trunk passed
          +1 mvnsite 0m 11s trunk passed
          +1 mvneclipse 0m 9s trunk passed
          +1 javadoc 0m 9s trunk passed
          +1 mvninstall 0m 7s the patch passed
          +1 compile 0m 6s the patch passed
          +1 javac 0m 6s the patch passed
          +1 mvnsite 0m 9s the patch passed
          +1 mvneclipse 0m 6s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          +1 javadoc 0m 7s the patch passed
          +1 unit 0m 6s hadoop-project in the patch passed.
          +1 asflicense 0m 15s The patch does not generate ASF License warnings.
          9m 35s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Issue HADOOP-13363
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12816966/HADOOP-13363.001.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml
          uname Linux 714e389ffa51 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / c62ae71
          Default Java 1.8.0_101
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10826/testReport/
          modules C: hadoop-project U: hadoop-project
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10826/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 18s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 7m 7s trunk passed +1 compile 0m 8s trunk passed +1 mvnsite 0m 11s trunk passed +1 mvneclipse 0m 9s trunk passed +1 javadoc 0m 9s trunk passed +1 mvninstall 0m 7s the patch passed +1 compile 0m 6s the patch passed +1 javac 0m 6s the patch passed +1 mvnsite 0m 9s the patch passed +1 mvneclipse 0m 6s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. +1 javadoc 0m 7s the patch passed +1 unit 0m 6s hadoop-project in the patch passed. +1 asflicense 0m 15s The patch does not generate ASF License warnings. 9m 35s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Issue HADOOP-13363 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12816966/HADOOP-13363.001.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml uname Linux 714e389ffa51 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / c62ae71 Default Java 1.8.0_101 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10826/testReport/ modules C: hadoop-project U: hadoop-project Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10826/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          stack stack added a comment -

          Did a ML discussion happen?

          pb3.1.0 is out. It runs in a 2.5.0 compatibility mode by default. Has some facility for saving on data copying that might be of interest in the NN. If upgrading, you need to run the newer protoc. Newer lib can't read the protos made by older protoc (IIRC). Newer protoc, in my experience, has no problem digesting pb 2.5.0 .proto files. The generated files are a little different, not consumable by the old protobuf lib.

          Would this be a problem? Old clients can talk to the new servers because of wire compatible. Is anyone consuming hadoop protos directly other than hadoop? Are hadoop proto files considered InterfaceAudience.Private or InterfaceAudience.Public? If the former, I could work on a patch for 3.0.0 (It'd be big but boring). Does Hadoop have Protobuf in its API anywhere (I can take a look but being lazy asking here first).

          Show
          stack stack added a comment - Did a ML discussion happen? pb3.1.0 is out. It runs in a 2.5.0 compatibility mode by default. Has some facility for saving on data copying that might be of interest in the NN. If upgrading, you need to run the newer protoc. Newer lib can't read the protos made by older protoc (IIRC). Newer protoc, in my experience, has no problem digesting pb 2.5.0 .proto files. The generated files are a little different, not consumable by the old protobuf lib. Would this be a problem? Old clients can talk to the new servers because of wire compatible. Is anyone consuming hadoop protos directly other than hadoop? Are hadoop proto files considered InterfaceAudience.Private or InterfaceAudience.Public? If the former, I could work on a patch for 3.0.0 (It'd be big but boring). Does Hadoop have Protobuf in its API anywhere (I can take a look but being lazy asking here first).
          Hide
          ozawa Tsuyoshi Ozawa added a comment -

          Thank you for the clarification, Steve. I got the point. This kind of "updating dependency work" is related to the classpath isolation work(HADOOP-13070), so I'd like to start discussion on ML.

          Show
          ozawa Tsuyoshi Ozawa added a comment - Thank you for the clarification, Steve. I got the point. This kind of "updating dependency work" is related to the classpath isolation work( HADOOP-13070 ), so I'd like to start discussion on ML.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          wire-level backward compatibility is a core feature of protobuf, I'm not worried there. What I am worried about is compile-compat, as that is what broke hadoop, hbase, etc, everything using protobuf 2.4. Google's internal build process is a clean, unified build of everything, so they don't have to worry about source level compatibliity

          Show
          stevel@apache.org Steve Loughran added a comment - wire-level backward compatibility is a core feature of protobuf, I'm not worried there. What I am worried about is compile-compat, as that is what broke hadoop, hbase, etc, everything using protobuf 2.4. Google's internal build process is a clean, unified build of everything, so they don't have to worry about source level compatibliity
          Hide
          ozawa Tsuyoshi Ozawa added a comment - - edited

          This merits a discussion on common-dev for visibility. There was a discussion before but no conclusion iirc.

          Sure.

          What is going to happen when code generated by protoc 2.5 tries to run against 2.6 library? Has anyone tested this? Because thats the metric of how traumatic this is going to be

          We must test it before merging it, of course. IIUC, protobuf is wire compatible between 2.5 and 2.6. I found a test suite which seems useful here: https://chromium.googlesource.com/external/github.com/google/protobuf/+/master/java/compatibility_tests/README.md

          We also need to check code-level compatibility.

          Show
          ozawa Tsuyoshi Ozawa added a comment - - edited This merits a discussion on common-dev for visibility. There was a discussion before but no conclusion iirc. Sure. What is going to happen when code generated by protoc 2.5 tries to run against 2.6 library? Has anyone tested this? Because thats the metric of how traumatic this is going to be We must test it before merging it, of course. IIUC, protobuf is wire compatible between 2.5 and 2.6. I found a test suite which seems useful here: https://chromium.googlesource.com/external/github.com/google/protobuf/+/master/java/compatibility_tests/README.md We also need to check code-level compatibility.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          What is going to happen when code generated by protoc 2.5 tries to run against 2.6 library? Has anyone tested this? Because thats the metric of how traumatic this is going to be

          Show
          stevel@apache.org Steve Loughran added a comment - What is going to happen when code generated by protoc 2.5 tries to run against 2.6 library? Has anyone tested this? Because thats the metric of how traumatic this is going to be
          Hide
          arpitagarwal Arpit Agarwal added a comment -

          This merits a discussion on common-dev for visibility. There was a discussion before but no conclusion iirc.

          I think it is fine as long as it does not break wire compatibility and we can shade the dependency, to be kind to downstream projects.

          Show
          arpitagarwal Arpit Agarwal added a comment - This merits a discussion on common-dev for visibility. There was a discussion before but no conclusion iirc. I think it is fine as long as it does not break wire compatibility and we can shade the dependency, to be kind to downstream projects.
          Hide
          aw Allen Wittenauer added a comment -

          We also need to update the Dockerfile to pull in protobuf 2.6.x, which may end up being tricky.

          Show
          aw Allen Wittenauer added a comment - We also need to update the Dockerfile to pull in protobuf 2.6.x, which may end up being tricky.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          Thanks Allen for reporting this and thanks Tsuyoshi for the initial patch. We need to update BUILDING.txt as well.

          Show
          ajisakaa Akira Ajisaka added a comment - Thanks Allen for reporting this and thanks Tsuyoshi for the initial patch. We need to update BUILDING.txt as well.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 18s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 mvninstall 6m 38s trunk passed
          +1 compile 0m 9s trunk passed
          +1 mvnsite 0m 11s trunk passed
          +1 mvneclipse 0m 9s trunk passed
          +1 javadoc 0m 9s trunk passed
          +1 mvninstall 0m 7s the patch passed
          +1 compile 0m 6s the patch passed
          +1 javac 0m 6s the patch passed
          +1 mvnsite 0m 8s the patch passed
          +1 mvneclipse 0m 7s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 2s The patch has no ill-formed XML file.
          +1 javadoc 0m 6s the patch passed
          +1 unit 0m 6s hadoop-project in the patch passed.
          +1 asflicense 0m 15s The patch does not generate ASF License warnings.
          9m 5s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12816966/HADOOP-13363.001.patch
          JIRA Issue HADOOP-13363
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml
          uname Linux 0f4f7012a335 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 932aed6
          Default Java 1.8.0_91
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9949/testReport/
          modules C: hadoop-project U: hadoop-project
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9949/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 18s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 6m 38s trunk passed +1 compile 0m 9s trunk passed +1 mvnsite 0m 11s trunk passed +1 mvneclipse 0m 9s trunk passed +1 javadoc 0m 9s trunk passed +1 mvninstall 0m 7s the patch passed +1 compile 0m 6s the patch passed +1 javac 0m 6s the patch passed +1 mvnsite 0m 8s the patch passed +1 mvneclipse 0m 7s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 2s The patch has no ill-formed XML file. +1 javadoc 0m 6s the patch passed +1 unit 0m 6s hadoop-project in the patch passed. +1 asflicense 0m 15s The patch does not generate ASF License warnings. 9m 5s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12816966/HADOOP-13363.001.patch JIRA Issue HADOOP-13363 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml uname Linux 0f4f7012a335 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 932aed6 Default Java 1.8.0_91 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9949/testReport/ modules C: hadoop-project U: hadoop-project Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9949/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          ozawa Tsuyoshi Ozawa added a comment -

          From CHANGES.txt: https://github.com/google/protobuf/blob/master/CHANGES.txt

          2.6.1

          Java

          • Updated DynamicMessage.setField() to handle repeated enum values
            correctly.
          • Fixed a bug that caused NullPointerException to be thrown when
            converting manually constructed FileDescriptorProto to
            FileDescriptor.

          2.6.0

          General
          General

          • Added oneofs(unions) feature. Fields in the same oneof will share
            memory and at most one field can be set at the same time. Use the
            oneof keyword to define a oneof like:
                  message SampleMessage {
                    oneof test_oneof {
                      string name = 4;
                      YourMessage sub_message = 9;
                    }
                  }
            
          • Files, services, enums, messages, methods and enum values can be marked
            as deprecated now.
          • Added Support for list values, including lists of messages, when
            parsing text-formatted protos in C++ and Java.
            For example: foo: [1, 2, 3]
            ...
            Java
          • writeTo() method in ByteString can now write a substring to an
            output stream. Added endWith() method for ByteString.
          • ByteString and ByteBuffer are now supported in CodedInputStream
            and CodedOutputStream.
          • java_generate_equals_and_hash can now be used with the LITE_RUNTIME.
          Show
          ozawa Tsuyoshi Ozawa added a comment - From CHANGES.txt: https://github.com/google/protobuf/blob/master/CHANGES.txt 2.6.1 Java Updated DynamicMessage.setField() to handle repeated enum values correctly. Fixed a bug that caused NullPointerException to be thrown when converting manually constructed FileDescriptorProto to FileDescriptor. 2.6.0 General General Added oneofs(unions) feature. Fields in the same oneof will share memory and at most one field can be set at the same time. Use the oneof keyword to define a oneof like: message SampleMessage { oneof test_oneof { string name = 4; YourMessage sub_message = 9; } } Files, services, enums, messages, methods and enum values can be marked as deprecated now. Added Support for list values, including lists of messages, when parsing text-formatted protos in C++ and Java. For example: foo: [1, 2, 3] ... Java writeTo() method in ByteString can now write a substring to an output stream. Added endWith() method for ByteString. ByteString and ByteBuffer are now supported in CodedInputStream and CodedOutputStream. java_generate_equals_and_hash can now be used with the LITE_RUNTIME.

            People

            • Assignee:
              ozawa Tsuyoshi Ozawa
              Reporter:
              aw Allen Wittenauer
            • Votes:
              1 Vote for this issue
              Watchers:
              33 Start watching this issue

              Dates

              • Created:
                Updated:

                Development