[HADOOP-12090] minikdc-related unit tests fail consistently on some platforms

• Type: Bug
• Status: Patch Available
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 2.7.0
• Component/s: kms, test
• Target Version/s: 2.8.3
• Assignee: Sangjin Lee
• Reporter: Sangjin Lee
• Created: 15/Jun/15 18:37
• Updated: 11/Sep/17 05:20

Description

On some platforms all unit tests that use minikdc fail consistently. Those tests include TestKMS, TestSaslDataTransfer, TestTimelineAuthenticationFilter, etc.

Typical failures on the unit tests:

java.lang.AssertionError: org.apache.hadoop.security.authentication.client.AuthenticationException: GSSException: No valid credentials provided (Mechanism level: Cannot get a KDC reply)
	at org.junit.Assert.fail(Assert.java:88)
	at org.apache.hadoop.crypto.key.kms.server.TestKMS$8$4.run(TestKMS.java:1154)
	at org.apache.hadoop.crypto.key.kms.server.TestKMS$8$4.run(TestKMS.java:1145)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1645)
	at org.apache.hadoop.crypto.key.kms.server.TestKMS.doAs(TestKMS.java:261)
	at org.apache.hadoop.crypto.key.kms.server.TestKMS.access$100(TestKMS.java:76)

The errors that cause this failure on the KDC server on the minikdc are a NullPointerException:

org.apache.mina.filter.codec.ProtocolDecoderException: java.lang.NullPointerException: message (Hexdump: ...)
	at org.apache.mina.filter.codec.ProtocolCodecFilter.messageReceived(ProtocolCodecFilter.java:234)
	at org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:434)
	at org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1200(DefaultIoFilterChain.java:48)
	at org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.messageReceived(DefaultIoFilterChain.java:802)
	at org.apache.mina.core.filterchain.IoFilterAdapter.messageReceived(IoFilterAdapter.java:120)
	at org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:434)
	at org.apache.mina.core.filterchain.DefaultIoFilterChain.fireMessageReceived(DefaultIoFilterChain.java:426)
	at org.apache.mina.core.polling.AbstractPollingIoProcessor.read(AbstractPollingIoProcessor.java:604)
	at org.apache.mina.core.polling.AbstractPollingIoProcessor.process(AbstractPollingIoProcessor.java:564)
	at org.apache.mina.core.polling.AbstractPollingIoProcessor.process(AbstractPollingIoProcessor.java:553)
	at org.apache.mina.core.polling.AbstractPollingIoProcessor.access$400(AbstractPollingIoProcessor.java:57)
	at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.run(AbstractPollingIoProcessor.java:892)
	at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:65)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NullPointerException: message
	at org.apache.mina.filter.codec.AbstractProtocolDecoderOutput.write(AbstractProtocolDecoderOutput.java:44)
	at org.apache.directory.server.kerberos.protocol.codec.MinaKerberosDecoder.decode(MinaKerberosDecoder.java:65)
	at org.apache.mina.filter.codec.ProtocolCodecFilter.messageReceived(ProtocolCodecFilter.java:224)
	... 15 more

Issue Links

• is duplicated by: HDFS-9746 Some Kerberos related tests intermittently fail.
• is related to: DIRSERVER-2071 MinaKerberosDecoder fails with NullPointerException if the kerberos message is split in multiple packets
• is related to: DIRSERVER-2074 small default TCP receive/send buffer size causing TCP packet fragmentation on some platforms