Agreed with flagging as an incompatible change.

I think this should be flagged as an incompatible change due to changing the type of some public members on IA.Public classes. any objections?

Thanks for your review and commit, Akira Ajisaka. It's my pleasure.

Github user wenxinhe closed the pull request at:

https://github.com/apache/hadoop/pull/251

+1, committed. Thank you very much, Wenxin He!

Hi Akira Ajisaka, a patch for branch-2 is ready. Please review, thanks.

-1 overall

This message was automatically generated.

-1 overall

This message was automatically generated.

patch for branch-2 upload:
Move commons logging APIs over to slf4j in hadoop-common in branch-2 and fix some checkstyle warnings.

PS: checkstyle report attached to explain why some extra changes are made. diff-checkstyle-hadoop-common-project.txt

GitHub user wenxinhe opened a pull request:

https://github.com/apache/hadoop/pull/251

HADOOP-14539. Move commons logging APIs over to slf4j in hadoop-common.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/wenxinhe/hadoop HADOOP-14539.hadoop-common.logging-branch-2

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/hadoop/pull/251.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #251

commit 53ee0fb11919169d952e1393d4f0749fd2771561
Author: wenxinhe <wenxin.he@gmail.com>
Date: 2017-07-18T11:06:25Z

HADOOP-14539. Move commons logging APIs over to slf4j in hadoop-common.

Github user wenxinhe closed the pull request at:

https://github.com/apache/hadoop/pull/246

Thanks, Akira Ajisaka.

would you create a patch for branch-2?

OK, I am testing the branch-2 patch and will upload later.

SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #12023 (See https://builds.apache.org/job/Hadoop-trunk-Commit/12023/)
HADOOP-14539. Move commons logging APIs over to slf4j in hadoop-common. (aajisaka: rev ccaf036662e22da14583942054898c99fa51dae5)

• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/OpensslCipher.java
• (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ha/ActiveStandbyElectorTestUtil.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/LdapGroupsMapping.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Client.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/JniBasedUnixGroupsNetgroupMappingWithFallback.java
• (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/http/TestServletFilter.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/metrics/RetryCacheMetrics.java
• (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/net/TestDNS.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/RPC.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/compress/snappy/SnappyDecompressor.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/FastByteComparisons.java
• (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/TestSequenceFile.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/ProtobufRpcEngine.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/UTF8.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/NativeCodeLoader.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/OpensslAesCtrCryptoCodec.java
• (edit) hadoop-common-project/hadoop-nfs/src/main/java/org/apache/hadoop/oncrpc/SimpleUdpServer.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/http/CrossOriginFilter.java
• (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestWinUtils.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/metrics/RpcMetrics.java
• (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/metrics2/impl/TestMetricsConfig.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/Trash.java
• (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestClasspath.java
• (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/loadGenerator/LoadGenerator.java
• (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/metrics2/lib/TestMutableMetrics.java
• (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/metrics2/util/TestMetricsCache.java
• (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/TestSetFile.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/compress/zlib/ZlibCompressor.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/concurrent/AsyncGetFuture.java
• (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestFindClass.java
• (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/net/unix/TestDomainSocketWatcher.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/concurrent/HadoopScheduledThreadPoolExecutor.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/DelegationTokenRenewer.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/unix/DomainSocketWatcher.java
• (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/net/TestSocketIOWithTimeout.java
• (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ha/TestHAAdmin.java
• (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/compress/TestCodec.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/jmx/JMXJsonServlet.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/compress/snappy/SnappyCompressor.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/compress/CompressionCodecFactory.java
• (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/test/MetricsAsserts.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/ThreadUtil.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/metrics/RpcDetailedMetrics.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/ScriptBasedMapping.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/GenericOptionsParser.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/ShellBasedIdMapping.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/tracing/TracerConfigurationManager.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/FailoverController.java
• (edit) hadoop-common-project/hadoop-nfs/src/main/java/org/apache/hadoop/portmap/Portmap.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/TrashPolicyDefault.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/BloomMapFile.java
• (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/test/MultithreadedTestUtil.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/random/OsSecureRandom.java
• (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestAsyncDiskService.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/FairCallQueue.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/sink/GraphiteSink.java
• (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/metrics2/impl/TestSinkQueue.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/nativeio/NativeIO.java
• (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/net/TestNetUtils.java
• (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestLightWeightGSet.java
• (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/http/TestHttpServerWebapps.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/NodeHealthScriptRunner.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/alias/AbstractJavaKeyStoreProvider.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/file/tfile/BCFile.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/AbstractFileSystem.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/retry/RetryUtils.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/ShutdownHookManager.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/JniBasedUnixGroupsNetgroupMapping.java
• (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/test/TestGenericTestUtils.java
• (edit) hadoop-common-project/hadoop-nfs/src/main/java/org/apache/hadoop/oncrpc/SimpleTcpClientHandler.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/JvmPauseMonitor.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/NodeFencer.java
• (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestProtoBufRpcServerHandoff.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/http/lib/StaticUserWebFilter.java
• (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ha/DummyHAService.java
• (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/nativeio/TestNativeIO.java
• (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestAsyncIPC.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/ErasureCodeNative.java
• (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/net/TestStaticMapping.java
• (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/authorize/TestAccessControlList.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/compress/bzip2/Bzip2Decompressor.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/file/tfile/TFileDumper.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/compress/bzip2/Bzip2Factory.java
• (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/http/TestHttpServer.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/lib/MutableMetricsFactory.java
• (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestSaslRPC.java
• (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestHarFileSystem.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/WhitelistBasedResolver.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Progress.java
• (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestGroupFallback.java
• (edit) hadoop-common-project/hadoop-nfs/src/main/java/org/apache/hadoop/oncrpc/RpcProgram.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/sink/ganglia/GangliaSink31.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/protocolPB/HAServiceProtocolServerSideTranslatorPB.java
• (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestRPCCompatibility.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/JceAesCtrCryptoCodec.java
• (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/http/TestGlobalFilter.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/AuthenticationWithProxyUserFilter.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/SaslRpcServer.java
• (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestRPC.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/util/MBeans.java
• (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/contract/AbstractBondedFSContract.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/ReconfigurableBase.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/Globber.java
• (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ha/MiniZKFCCluster.java
• (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/compress/TestCompressionStreamReuse.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/Groups.java
• (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/crypto/TestCryptoCodec.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/GSet.java
• (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/token/delegation/TestDelegationToken.java
• (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/TestArrayFile.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/random/OpensslSecureRandom.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/unix/DomainSocket.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/HarFileSystem.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/compress/CodecPool.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/concurrent/HadoopThreadPoolExecutor.java
• (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/TestWritableUtils.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/compress/zlib/BuiltInZlibDeflater.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/ReadaheadPool.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/WritableRpcEngine.java
• (edit) hadoop-common-project/hadoop-nfs/src/main/java/org/apache/hadoop/mount/MountdBase.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/SaslInputStream.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/sftp/SFTPFileSystem.java
• (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestWeightedRoundRobinMultiplexer.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/VersionInfo.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/JniBasedUnixGroupsMappingWithFallback.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/WeightedRoundRobinMultiplexer.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/file/tfile/Compression.java
• (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFsShellTouch.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/ProviderUtils.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/service/AbstractService.java
• (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestIPCServerResponder.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/SaslRpcClient.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/Command.java
• (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestCompositeGroupMapping.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/SecurityUtil.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FSInputChecker.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/TableMapping.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/lib/MetricsSourceBuilder.java
• (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestRpcServerHandoff.java
• (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/alias/TestCredentialProviderFactory.java
• (edit) hadoop-common-project/hadoop-nfs/src/main/java/org/apache/hadoop/oncrpc/SimpleTcpServer.java
• (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestNativeCodeLoader.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/JniBasedUnixGroupsMapping.java
• (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/metrics2/impl/TestMetricsSystemImpl.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/impl/MBeanInfoBuilder.java
• (edit) hadoop-common-project/hadoop-nfs/src/main/java/org/apache/hadoop/oncrpc/RegistrationClient.java
• (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/nativeio/TestSharedFileDescriptorFactory.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/CompositeGroupsMapping.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/CombinedIPWhiteList.java
• (edit) hadoop-common-project/hadoop-nfs/src/main/java/org/apache/hadoop/portmap/RpcProgramPortmap.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/NetUtils.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/SysInfoWindows.java
• (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/service/TestCompositeService.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/compress/DefaultCodec.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/token/delegation/AbstractDelegationTokenSecretManager.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/FileBasedIPList.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java
• (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/http/TestPathFilter.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/permission/FsPermission.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/compress/zlib/ZlibFactory.java
• (edit) hadoop-common-project/hadoop-nfs/src/main/java/org/apache/hadoop/oncrpc/security/SecurityHandler.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/ReconfigurationServlet.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/IntrusiveCollection.java
• (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestRPCServerShutdown.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/authorize/ServiceAuthorizationManager.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FsShell.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/SequenceFile.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/HostsFileReader.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/HAAdmin.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/lib/MutableRatesWithAggregation.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/ShutdownThreadsHelper.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/util/MetricsCache.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/serializer/SerializationFactory.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/nativeio/SharedFileDescriptorFactory.java
• (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFsShellCopy.java
• (edit) hadoop-common-project/hadoop-nfs/src/main/java/org/apache/hadoop/nfs/nfs3/FileHandle.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/token/Token.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/DNS.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/service/ServiceOperations.java
• (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestIPC.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/HttpCrossOriginFilterInitializer.java
• (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFileStatus.java
• (edit) hadoop-common-project/hadoop-nfs/src/main/java/org/apache/hadoop/oncrpc/RpcUtil.java
• (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/authorize/TestProxyUsers.java
• (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/http/TestHttpServerLogs.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/MachineList.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FsShellPermissions.java
• (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/http/TestSSLHttpServer.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/ftp/FTPFileSystem.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/SshFenceByTcpPort.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/impl/MetricsConfig.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/concurrent/ExecutorHelper.java
• (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/crypto/CryptoStreamsTestBase.java
• (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ha/TestHealthMonitor.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/lib/MethodMetric.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/AsyncDiskService.java
• (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/test/TestJUnitSetup.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/RefreshRegistry.java
• (edit) hadoop-common-project/hadoop-nfs/src/main/java/org/apache/hadoop/nfs/nfs3/Nfs3Base.java
• (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/metrics2/impl/TestGangliaMetrics.java
• (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/net/unix/TestDomainSocket.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/SysInfoLinux.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/ApplicationClassLoader.java
• (edit) hadoop-common-project/hadoop-nfs/src/main/java/org/apache/hadoop/nfs/NfsExports.java
• (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestSignalLogger.java
• (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/http/TestHttpServerWithSpengo.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/compress/bzip2/Bzip2Compressor.java
• (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFileContext.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/ChecksumFs.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileUtil.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/sftp/SFTPConnectionPool.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/RetryCache.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/service/LoggingStateChangeListener.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/token/DtFileOperations.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/http/HttpServer2.java
• (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFsShellReturnCode.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileContext.java
• (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestIdentityHashStore.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/ssl/ReloadingX509TrustManager.java
• (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/http/resource/JerseyResource.java
• (edit) hadoop-common-project/hadoop-nfs/src/main/java/org/apache/hadoop/oncrpc/RpcCall.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/MapFile.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/IOUtils.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/file/tfile/TFile.java
• (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFileUtil.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/lib/MutableRates.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/ShellBasedUnixGroupsNetgroupMapping.java
• (edit) hadoop-common-project/hadoop-nfs/src/main/java/org/apache/hadoop/oncrpc/security/Credentials.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/impl/MetricsSystemImpl.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/Credentials.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/retry/RetryPolicies.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/sink/ganglia/GangliaSink30.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/impl/MetricsSourceAdapter.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/impl/MetricsSinkAdapter.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/ActiveStandbyElector.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/token/DtUtilShell.java
• (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestServer.java
• (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/service/TestServiceLifecycle.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/ZKFailoverController.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/SocketIOWithTimeout.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/sink/ganglia/AbstractGangliaSink.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/CodecUtil.java
• (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/FCStatisticsBaseTest.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/compress/lz4/Lz4Compressor.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/LocalDirAllocator.java
• (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestGroupsCaching.java
• (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestDoAsEffectiveUser.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java
• (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestShellBasedUnixGroupsMapping.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/service/CompositeService.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/compress/lz4/Lz4Decompressor.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/HealthMonitor.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/CallQueueManager.java
• (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/TestDefaultStringifier.java
• (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestLightWeightResizableGSet.java
• (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/ssl/FileBasedKeyStoresFactory.java
• (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/net/ServerSocketUtil.java

Committed this to trunk. Hi Wenxin He, would you create a patch for branch-2?

Thanks Wenxin He! Sorry I was wrong and I missed something. I ran all the failed tests again with/without your patch and the following three tests failed regardless of the patch.

• TestFSImage
• TestContainerManagerSecurity
• TestMRJobs

Now I'm +1 for your patch. Checking this in.

Thanks for running all tests, Akira Ajisaka.

I tried to rerun these failed tests, but did not find the same NoSuchFieldError.

Also, I checked the code, found LOG in LightWeightGSet.computeCapacity(LightWeightGSet.java:395) is OK. It is declared in org.apache.hadoop.util.GSet.

Did I miss something?

The following is how I ran the tests:

1. checkout the latest trunk

   commit 02b141ac6059323ec43e472ca36dc570fdca386f
   Author: Anu Engineer <aengineer@apache.org>
   Date:   Sun Jul 16 10:59:34 2017 -0700
   
       HDFS-11786. Add support to make copyFromLocal multi threaded. Contributed by Mukul Kumar Singh.
   

2. compile and run the specified tests

   mvn clean install -DskipTests -Dmaven.javadoc.skip=true
   mvn -f hadoop-hdfs-project/hadoop-hdfs/pom.xml test -Dtest=TestLazyPersistReplicaRecovery,TestHAAppend,TestFSImage -Dmaven.test.failure.ignore=true
   

   Three tests failed which maybe caused by my environment:

   -------------------------------------------------------
    T E S T S
   -------------------------------------------------------
   
   -------------------------------------------------------
    T E S T S
   -------------------------------------------------------
   Running org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaRecovery
   Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 20.112 sec - in 
   
   org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaRecovery
   Running org.apache.hadoop.hdfs.server.namenode.TestFSImage
   Tests run: 14, Failures: 0, Errors: 3, Skipped: 0, Time elapsed: 65.195 sec <<< FAILURE! - in 
   
   org.apache.hadoop.hdfs.server.namenode.TestFSImage
   testHasNonEcBlockUsingStripedIDForLoadUCFile(org.apache.hadoop.hdfs.server.namenode.TestFSImage)  Time elapsed: 3.34 
   
   sec  <<< ERROR!
   java.lang.IllegalStateException: failed to create a child event loop
   	at sun.nio.ch.EPollArrayWrapper.epollCreate(Native Method)
   	at sun.nio.ch.EPollArrayWrapper.<init>(EPollArrayWrapper.java:130)
   	at sun.nio.ch.EPollSelectorImpl.<init>(EPollSelectorImpl.java:69)
   	at sun.nio.ch.EPollSelectorProvider.openSelector(EPollSelectorProvider.java:36)
   	at io.netty.channel.nio.NioEventLoop.openSelector(NioEventLoop.java:126)
   	at io.netty.channel.nio.NioEventLoop.<init>(NioEventLoop.java:120)
   	at io.netty.channel.nio.NioEventLoopGroup.newChild(NioEventLoopGroup.java:87)
   	at io.netty.util.concurrent.MultithreadEventExecutorGroup.<init>(MultithreadEventExecutorGroup.java:64)
   	at io.netty.channel.MultithreadEventLoopGroup.<init>(MultithreadEventLoopGroup.java:49)
   	at io.netty.channel.nio.NioEventLoopGroup.<init>(NioEventLoopGroup.java:61)
   	at io.netty.channel.nio.NioEventLoopGroup.<init>(NioEventLoopGroup.java:52)
   	at io.netty.channel.nio.NioEventLoopGroup.<init>(NioEventLoopGroup.java:44)
   	at io.netty.channel.nio.NioEventLoopGroup.<init>(NioEventLoopGroup.java:36)
   	at org.apache.hadoop.hdfs.server.datanode.web.DatanodeHttpServer.<init>(DatanodeHttpServer.java:131)
   	at org.apache.hadoop.hdfs.server.datanode.DataNode.startInfoServer(DataNode.java:954)
   	at org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:1402)
   	at org.apache.hadoop.hdfs.server.datanode.DataNode.<init>(DataNode.java:497)
   	at org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:2752)
   	at org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:2655)
   	at org.apache.hadoop.hdfs.MiniDFSCluster.startDataNodes(MiniDFSCluster.java:1621)
   	at org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.java:868)
   	at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:491)
   	at org.apache.hadoop.hdfs.MiniDFSCluster$Builder.build(MiniDFSCluster.java:450)
   	at org.apache.hadoop.hdfs.server.namenode.TestFSImage.testHasNonEcBlockUsingStripedIDForLoadUCFile
   
   (TestFSImage.java:625)
   
   testSupportBlockGroup(org.apache.hadoop.hdfs.server.namenode.TestFSImage)  Time elapsed: 3.677 sec  <<< ERROR!
   java.lang.IllegalStateException: failed to create a child event loop
   	at sun.nio.ch.IOUtil.makePipe(Native Method)
   	at sun.nio.ch.EPollSelectorImpl.<init>(EPollSelectorImpl.java:65)
   	at sun.nio.ch.EPollSelectorProvider.openSelector(EPollSelectorProvider.java:36)
   	at io.netty.channel.nio.NioEventLoop.openSelector(NioEventLoop.java:126)
   	at io.netty.channel.nio.NioEventLoop.<init>(NioEventLoop.java:120)
   	at io.netty.channel.nio.NioEventLoopGroup.newChild(NioEventLoopGroup.java:87)
   	at io.netty.util.concurrent.MultithreadEventExecutorGroup.<init>(MultithreadEventExecutorGroup.java:64)
   	at io.netty.channel.MultithreadEventLoopGroup.<init>(MultithreadEventLoopGroup.java:49)
   	at io.netty.channel.nio.NioEventLoopGroup.<init>(NioEventLoopGroup.java:61)
   	at io.netty.channel.nio.NioEventLoopGroup.<init>(NioEventLoopGroup.java:52)
   	at io.netty.channel.nio.NioEventLoopGroup.<init>(NioEventLoopGroup.java:44)
   	at io.netty.channel.nio.NioEventLoopGroup.<init>(NioEventLoopGroup.java:36)
   	at org.apache.hadoop.hdfs.server.datanode.web.DatanodeHttpServer.<init>(DatanodeHttpServer.java:131)
   	at org.apache.hadoop.hdfs.server.datanode.DataNode.startInfoServer(DataNode.java:954)
   	at org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:1402)
   	at org.apache.hadoop.hdfs.server.datanode.DataNode.<init>(DataNode.java:497)
   	at org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:2752)
   	at org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:2655)
   	at org.apache.hadoop.hdfs.MiniDFSCluster.startDataNodes(MiniDFSCluster.java:1621)
   	at org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.java:868)
   	at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:491)
   	at org.apache.hadoop.hdfs.MiniDFSCluster$Builder.build(MiniDFSCluster.java:450)
   	at org.apache.hadoop.hdfs.server.namenode.TestFSImage.testSupportBlockGroup(TestFSImage.java:469)
   
   testCompression(org.apache.hadoop.hdfs.server.namenode.TestFSImage)  Time elapsed: 21.451 sec  <<< ERROR!
   java.io.IOException: Failed to save in any storage directories while saving namespace.
   	at org.apache.hadoop.hdfs.server.namenode.FSImage.saveFSImageInAllDirs(FSImage.java:1174)
   	at org.apache.hadoop.hdfs.server.namenode.FSImage.saveFSImageInAllDirs(FSImage.java:1131)
   	at org.apache.hadoop.hdfs.server.namenode.FSImage.format(FSImage.java:169)
   	at org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:1153)
   	at org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:401)
   	at org.apache.hadoop.hdfs.DFSTestUtil.formatNameNode(DFSTestUtil.java:243)
   	at org.apache.hadoop.hdfs.MiniDFSCluster.configureNameService(MiniDFSCluster.java:1029)
   	at org.apache.hadoop.hdfs.MiniDFSCluster.createNameNodesAndSetConf(MiniDFSCluster.java:915)
   	at org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.java:847)
   	at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:491)
   	at org.apache.hadoop.hdfs.MiniDFSCluster$Builder.build(MiniDFSCluster.java:450)
   	at org.apache.hadoop.hdfs.server.namenode.TestFSImage.testPersistHelper(TestFSImage.java:109)
   	at org.apache.hadoop.hdfs.server.namenode.TestFSImage.setCompressCodec(TestFSImage.java:103)
   	at org.apache.hadoop.hdfs.server.namenode.TestFSImage.testCompression(TestFSImage.java:97)
   
   Running org.apache.hadoop.hdfs.server.namenode.ha.TestHAAppend
   Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.793 sec - in 
   
   org.apache.hadoop.hdfs.server.namenode.ha.TestHAAppend
   
   Results :
   
   Tests in error: 
     TestFSImage.testHasNonEcBlockUsingStripedIDForLoadUCFile:625 » IllegalState fa...
     TestFSImage.testSupportBlockGroup:469 » IllegalState failed to create a child ...
     TestFSImage.testCompression:97->setCompressCodec:103->testPersistHelper:109 » IO
   
   Tests run: 17, Failures: 0, Errors: 3, Skipped: 0
   

3. then, apply the patch, recompile and run the specified tests, the result is the same as before patched(the three test failures):

   git am HADOOP-14539.003.patch
   mvn clean install -DskipTests -Dmaven.javadoc.skip=true
   mvn -f hadoop-hdfs-project/hadoop-hdfs/pom.xml test -Dtest=TestLazyPersistReplicaRecovery,TestHAAppend,TestFSImage -Dmaven.test.failure.ignore=true
   

   -------------------------------------------------------
    T E S T S
   -------------------------------------------------------
   
   -------------------------------------------------------
    T E S T S
   -------------------------------------------------------
   Running org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaRecovery
   Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 19.407 sec - in org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaRecovery
   Running org.apache.hadoop.hdfs.server.namenode.TestFSImage
   Tests run: 14, Failures: 0, Errors: 3, Skipped: 0, Time elapsed: 58.457 sec <<< FAILURE! - in org.apache.hadoop.hdfs.server.namenode.TestFSImage
   testHasNonEcBlockUsingStripedIDForLoadUCFile(org.apache.hadoop.hdfs.server.namenode.TestFSImage)  Time elapsed: 1.476 sec  <<< ERROR!
   org.apache.hadoop.util.DiskChecker$DiskErrorException: Too many failed volumes - current valid volumes: 1, volumes configured: 2, volumes failed: 1, volume failures tolerated: 0
   	at org.apache.hadoop.hdfs.server.datanode.checker.StorageLocationChecker.check(StorageLocationChecker.java:220)
   	at org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:2745)
   	at org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:2655)
   	at org.apache.hadoop.hdfs.MiniDFSCluster.startDataNodes(MiniDFSCluster.java:1621)
   	at org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.java:868)
   	at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:491)
   	at org.apache.hadoop.hdfs.MiniDFSCluster$Builder.build(MiniDFSCluster.java:450)
   	at org.apache.hadoop.hdfs.server.namenode.TestFSImage.testHasNonEcBlockUsingStripedIDForLoadUCFile(TestFSImage.java:625)
   
   testSupportBlockGroup(org.apache.hadoop.hdfs.server.namenode.TestFSImage)  Time elapsed: 4.068 sec  <<< ERROR!
   java.lang.IllegalStateException: failed to create a child event loop
   	at sun.nio.ch.IOUtil.makePipe(Native Method)
   	at sun.nio.ch.EPollSelectorImpl.<init>(EPollSelectorImpl.java:65)
   	at sun.nio.ch.EPollSelectorProvider.openSelector(EPollSelectorProvider.java:36)
   	at io.netty.channel.nio.NioEventLoop.openSelector(NioEventLoop.java:126)
   	at io.netty.channel.nio.NioEventLoop.<init>(NioEventLoop.java:120)
   	at io.netty.channel.nio.NioEventLoopGroup.newChild(NioEventLoopGroup.java:87)
   	at io.netty.util.concurrent.MultithreadEventExecutorGroup.<init>(MultithreadEventExecutorGroup.java:64)
   	at io.netty.channel.MultithreadEventLoopGroup.<init>(MultithreadEventLoopGroup.java:49)
   	at io.netty.channel.nio.NioEventLoopGroup.<init>(NioEventLoopGroup.java:61)
   	at io.netty.channel.nio.NioEventLoopGroup.<init>(NioEventLoopGroup.java:52)
   	at io.netty.channel.nio.NioEventLoopGroup.<init>(NioEventLoopGroup.java:44)
   	at io.netty.channel.nio.NioEventLoopGroup.<init>(NioEventLoopGroup.java:36)
   	at org.apache.hadoop.hdfs.server.datanode.web.DatanodeHttpServer.<init>(DatanodeHttpServer.java:131)
   	at org.apache.hadoop.hdfs.server.datanode.DataNode.startInfoServer(DataNode.java:954)
   	at org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:1402)
   	at org.apache.hadoop.hdfs.server.datanode.DataNode.<init>(DataNode.java:497)
   	at org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:2752)
   	at org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:2655)
   	at org.apache.hadoop.hdfs.MiniDFSCluster.startDataNodes(MiniDFSCluster.java:1621)
   	at org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.java:868)
   	at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:491)
   	at org.apache.hadoop.hdfs.MiniDFSCluster$Builder.build(MiniDFSCluster.java:450)
   	at org.apache.hadoop.hdfs.server.namenode.TestFSImage.testSupportBlockGroup(TestFSImage.java:469)
   
   testCompression(org.apache.hadoop.hdfs.server.namenode.TestFSImage)  Time elapsed: 21.234 sec  <<< ERROR!
   java.io.IOException: Failed to save in any storage directories while saving namespace.
   	at org.apache.hadoop.hdfs.server.namenode.FSImage.saveFSImageInAllDirs(FSImage.java:1174)
   	at org.apache.hadoop.hdfs.server.namenode.FSImage.saveFSImageInAllDirs(FSImage.java:1131)
   	at org.apache.hadoop.hdfs.server.namenode.FSImage.format(FSImage.java:169)
   	at org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:1153)
   	at org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:401)
   	at org.apache.hadoop.hdfs.DFSTestUtil.formatNameNode(DFSTestUtil.java:243)
   	at org.apache.hadoop.hdfs.MiniDFSCluster.configureNameService(MiniDFSCluster.java:1029)
   	at org.apache.hadoop.hdfs.MiniDFSCluster.createNameNodesAndSetConf(MiniDFSCluster.java:915)
   	at org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.java:847)
   	at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:491)
   	at org.apache.hadoop.hdfs.MiniDFSCluster$Builder.build(MiniDFSCluster.java:450)
   	at org.apache.hadoop.hdfs.server.namenode.TestFSImage.testPersistHelper(TestFSImage.java:109)
   	at org.apache.hadoop.hdfs.server.namenode.TestFSImage.setCompressCodec(TestFSImage.java:103)
   	at org.apache.hadoop.hdfs.server.namenode.TestFSImage.testCompression(TestFSImage.java:97)
   
   Running org.apache.hadoop.hdfs.server.namenode.ha.TestHAAppend
   Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.828 sec - in org.apache.hadoop.hdfs.server.namenode.ha.TestHAAppend
   
   Results :
   
   Tests in error: 
     TestFSImage.testHasNonEcBlockUsingStripedIDForLoadUCFile:625 » DiskError Too m...
     TestFSImage.testSupportBlockGroup:469 » IllegalState failed to create a child ...
     TestFSImage.testCompression:97->setCompressCodec:103->testPersistHelper:109 » IO
   
   Tests run: 17, Failures: 0, Errors: 3, Skipped: 0
   
   [ERROR] There are test failures.
   
   Please refer to /home/hewenxin/Codebase/wenxinhe/hadoop/hadoop-hdfs-project/hadoop-hdfs/target/surefire-reports for the individual test results.
   [INFO] 
   [INFO] --- maven-antrun-plugin:1.7:run (hdfs-test-bats-driver) @ hadoop-hdfs ---
   [INFO] Executing tasks
   
   main:
        [exec] 
        [exec] 
        [exec] ERROR: bats not installed. Skipping bash tests.
        [exec] ERROR: Please install bats as soon as possible.
        [exec] 
        [exec] 
   [INFO] Executed tasks
   [INFO] ------------------------------------------------------------------------
   [INFO] BUILD SUCCESS
   [INFO] ------------------------------------------------------------------------
   [INFO] Total time: 01:44 min
   [INFO] Finished at: 2017-07-17T11:35:02+08:00
   [INFO] Final Memory: 41M/1477M
   [INFO] ------------------------------------------------------------------------
   

I ran all the tests and some HDFS tests (TestLazyPersistReplicaRecovery, TestHAAppend, TestFSImage) failed by NoSuchFieldError:

Running org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaRecovery
Tests run: 2, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 2.663 sec <<< FAILURE! - in org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaRecovery
testDnRestartWithSavedReplicas(org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaRecovery)  Time elapsed: 1.673 sec  <<< ERROR!
java.lang.NoSuchFieldError: LOG
	at org.apache.hadoop.util.LightWeightGSet.computeCapacity(LightWeightGSet.java:395)
	at org.apache.hadoop.util.LightWeightGSet.computeCapacity(LightWeightGSet.java:362)
	at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.<init>(BlockManager.java:442)
	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:778)
	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:718)
	at org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:1146)
	at org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:401)
	at org.apache.hadoop.hdfs.DFSTestUtil.formatNameNode(DFSTestUtil.java:243)
	at org.apache.hadoop.hdfs.MiniDFSCluster.configureNameService(MiniDFSCluster.java:1029)
	at org.apache.hadoop.hdfs.MiniDFSCluster.createNameNodesAndSetConf(MiniDFSCluster.java:915)
	at org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.java:847)
	at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:491)
	at org.apache.hadoop.hdfs.MiniDFSCluster$Builder.build(MiniDFSCluster.java:450)
	at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.LazyPersistTestCase.startUpCluster(LazyPersistTestCase.java:316)
	at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.LazyPersistTestCase$ClusterWithRamDiskBuilder.build(LazyPersistTestCase.java:415)
	at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaRecovery.testDnRestartWithSavedReplicas(TestLazyPersistReplicaRecovery.java:36)

This failure is related to the change. Would you fix this?

-1 javac 10m 7s root generated 9 new + 1306 unchanged - 52 fixed = 1315 total (was 1358)

9 new javac warnings are in hadoop-hdfs and caused by deprecated method setLogLevel(Logger,Level)

-0 checkstyle 1m 49s hadoop-common-project: The patch generated 7 new + 5116 unchanged - 59 fixed = 5123 total (was 5175)

7 new checkstyle issues leave unchanged intentionally.

-1 unit 7m 57s hadoop-common in the patch failed.

3 test failures are unrelated.

-1 overall

This message was automatically generated.

003.patch:
rebase and skip the commit that fixed in HADOOP-14638.

-1 overall

This message was automatically generated.

002.patch:

1. revert StreamPumper and related codes that are fixed in separate issue HADOOP-14638.
2. fix checkstyle issues: Indentation, LineLength.
3. fix checkstyle issues: StaticVariableName.
   1. add final modifier to LOG to make them real constant.
   2. leave LOG in ShellCommandFencer unchanged for it's reassigned in tests.
4. fix checkstyle issues: MemberName.
   1. add static modifier to LOG.
   2. leave LOG in AbstractGangliaSink, GangliaSink30 and GangliaSink31 unchanged for they're real instance member.
5. fix checkstyle issues: RedundantModifier, ConstantName.

-1 overall

This message was automatically generated.

I commented one minor nit in the GitHub PR.

Thanks, I've fixed it.

Let me see the result of the precommit job.

I found a test failure caused by this patch and filed a separate issue HADOOP-14638 to fix it.
Would you please to review this HADOOP-14638 first?

Github user wenxinhe commented on a diff in the pull request:

https://github.com/apache/hadoop/pull/246#discussion_r126337274

— Diff: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/SecurityUtil.java —
@@ -53,6 +51,8 @@

//this will need to be replaced someday when there is a suitable replacement
+import org.slf4j.Logger;
— End diff –

OK, fix it in d04928e.

Mostly looks good to me. I commented one minor nit in the GitHub PR. Let me see the result of the precommit job.

Github user aajisaka commented on a diff in the pull request:

https://github.com/apache/hadoop/pull/246#discussion_r126333782

— Diff: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/SecurityUtil.java —
@@ -53,6 +51,8 @@

//this will need to be replaced someday when there is a suitable replacement
+import org.slf4j.Logger;
— End diff –

Would you move the new imports above the comment "this will need to ...."?

Since the change is big, it's divided into several commit steps:

Step 1: (521ac19f9226ea271c239b9b7345d64d82ab35d4)
simply replace all org.apache.commons.logging.Log to org.slf4j.Logger where available.

Step 2: (a86f2b25202808f1e4ef2b6ce214f63dfa75e967)
change org.apache.commons.logging.LOG.fatal() to org.slf4j.LoggerLOG.error() for api incompatibility.

Step 3: (fa9af169900f3515e65814ef7ca1c98914e90715)
change org.apache.commons.logging.Log.debug(Object) to org.slf4j.Logger.debug(String) for api incompatibility.

Step 4: (7af274b3e89da963691272335256004ebf9f1b83)
handle a special case in DelegationTokenRenewer: change LOG.debug(ie.getStackTrace()) to LOG.debug("Exception in removeRenewAction: ", ie)

Step 5: (7af274b3e89da963691272335256004ebf9f1b83)
handle a special case in AbstractService: change LOG.debug("noteFailure " + exception, null) to LOG.debug("noteFailure " + exception, (Throwable) null)

Step 6: (4ce2b824e77590407f77f326f16c39175aceccbf)
change IOUtils.cleanup to IOUtils.cleanupWithLogger

Step 7: (d82797fb88dbea12ffa0db926b894cc9123e444f)
revert FileSystem.LOG for it's referenced by hadoop-yarn

Step 8: (f0ffa503fb135d84e542ecea80cb3d6224983cd3)
change SignalLogger.INSTANCE.register(LOG) to SignalLogger.INSTANCE.register(LogAdapter.create(LOG))

GitHub user wenxinhe opened a pull request:

https://github.com/apache/hadoop/pull/246

HADOOP-14539. Move commons logging APIs over to slf4j in hadoop-common.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/wenxinhe/hadoop HADOOP-14539.hadoop-common.logging

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/hadoop/pull/246.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #246

commit 521ac19f9226ea271c239b9b7345d64d82ab35d4
Author: wenxinhe <wenxin.he@gmail.com>
Date: 2017-07-01T02:12:34Z

HADOOP-14539. Move commons logging APIs over to slf4j in hadoop-common.

simple replace: org.apache.commons.logging.Log -> org.slf4j.Logger

commit a86f2b25202808f1e4ef2b6ce214f63dfa75e967
Author: wenxinhe <wenxin.he@gmail.com>
Date: 2017-07-06T03:54:07Z

HADOOP-14539. Move commons logging APIs over to slf4j in hadoop-common.

change: LOG.fatal() -> LOG.error()

commit fa9af169900f3515e65814ef7ca1c98914e90715
Author: wenxinhe <wenxin.he@gmail.com>
Date: 2017-07-01T09:24:57Z

HADOOP-14539. Move commons logging APIs over to slf4j in hadoop-common.

change: org.apache.commons.logging.Log.debug(Object)
->org.slf4j.Logger.debug(String)

commit 7af274b3e89da963691272335256004ebf9f1b83
Author: wenxinhe <wenxin.he@gmail.com>
Date: 2017-07-03T06:34:31Z

HADOOP-14539. Move commons logging APIs over to slf4j in hadoop-common.

change: org.apache.commons.logging.Log.debug(Object, (Throwable) null)
-> org.slf4j.Logger.debug(String, (Throwable) null)

commit fd6f14dcdbf36b403bd5efa114436d945959ac56
Author: wenxinhe <wenxin.he@gmail.com>
Date: 2017-07-06T08:03:43Z

HADOOP-14539. Move commons logging APIs over to slf4j in hadoop-common.

change: org.apache.commons.logging.Log.debug(Object)
-> org.slf4j.Logger.debug(String, Throwable)

commit 4ce2b824e77590407f77f326f16c39175aceccbf
Author: wenxinhe <wenxin.he@gmail.com>
Date: 2017-07-06T08:54:37Z

HADOOP-14539. Move commons logging APIs over to slf4j in hadoop-common.

change: IOUtils.cleanup -> IOUtils.cleanupWithLogger

commit d82797fb88dbea12ffa0db926b894cc9123e444f
Author: wenxinhe <wenxin.he@gmail.com>
Date: 2017-07-06T11:24:28Z

HADOOP-14539. Move commons logging APIs over to slf4j in hadoop-common.

revert FileSystem.LOG for it's referenced by hadoop-yarn

commit f0ffa503fb135d84e542ecea80cb3d6224983cd3
Author: wenxinhe <wenxin.he@gmail.com>
Date: 2017-07-07T02:58:27Z

HADOOP-14539. Move commons logging APIs over to slf4j in hadoop-common.

change: SignalLogger.INSTANCE.register(LOG)
-> SignalLogger.INSTANCE.register(LogAdapter.create(LOG))

Hi, Akira Ajisaka, I'm not sure what's the situation of this issue now. I created a pull request, hope It may help.

Filed HDFS-11992 for InstrumentedLock and InstrumentedWriteLock.