Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-9745

TestSecureNNWithQJM#testSecureMode sometimes fails with timeouts

    Details

    • Type: Test
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 2.7.4, 3.0.0-alpha1
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      TestSecureNNWithQJM#testSecureMode fails intermittently. For most of the case, it timeouts.
      In a 0.5%~1% probability, it fails with a more sophisticated error.

        Issue Links

          Activity

          Hide
          xiaochen Xiao Chen added a comment -

          Here is the more sophisticated error. I haven't root caused it, but kerberos seems to be involved. I've also seen this NPE in TestKMS#testACLs failures..

          2016-02-01 06:41:47,944 INFO  namenode.FSImage (FSImage.java:loadEdits(832)) - Reading org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream@12e8c783 expecting start txid #1
          2016-02-01 06:41:47,944 INFO  namenode.FSImage (FSEditLogLoader.java:loadFSEdits(139)) - Start loading edits file https://localhost:55326/getJournal?jid=myjournal&segmentTxId=1&storageInfo=-60%3A1226055383%3A0%3AtestClusterID, https://localhost:48398/getJournal?jid=myjournal&segmentTxId=1&storageInfo=-60%3A1226055383%3A0%3AtestClusterID
          2016-02-01 06:41:47,945 INFO  namenode.EditLogInputStream (RedundantEditLogInputStream.java:nextOp(176)) - Fast-forwarding stream 'https://localhost:55326/getJournal?jid=myjournal&segmentTxId=1&storageInfo=-60%3A1226055383%3A0%3AtestClusterID, https://localhost:48398/getJournal?jid=myjournal&segmentTxId=1&storageInfo=-60%3A1226055383%3A0%3AtestClusterID' to transaction ID 1
          2016-02-01 06:41:47,945 INFO  namenode.EditLogInputStream (RedundantEditLogInputStream.java:nextOp(176)) - Fast-forwarding stream 'https://localhost:55326/getJournal?jid=myjournal&segmentTxId=1&storageInfo=-60%3A1226055383%3A0%3AtestClusterID' to transaction ID 1
          2016-02-01 06:41:48,131 ERROR protocol.KerberosProtocolHandler (KerberosProtocolHandler.java:exceptionCaught(157)) - /127.0.0.1:46417 EXCEPTION
          org.apache.mina.filter.codec.ProtocolDecoderException: java.lang.NullPointerException: message (Hexdump: 00 00 02 45 6C 82 02 41 30 82 02 3D A1 03 02 01 05 A2 03 02 01 0C A3 82 01 C4 30 82 01 C0 30 82 01 BC A1 03 02 01 01 A2 82 01 B3 04 82 01 AF 6E 82 01 AB 30 82 01 A7 A0 03 02 01 05 A1 03 02 01 0E A2 07 03 05 00 00 00 00 00 A3 81 F6 61 81 F3 30 81 F0 A0 03 02 01 05 A1 0D 1B 0B 45 58 41 4D 50 4C 45 2E 43 4F 4D A2 20 30 1E A0 03 02 01 02 A1 17 30 15 1B 06 6B 72 62 74 67 74 1B 0B 45 58 41 4D 50 4C 45 2E 43 4F 4D A3 81 B7 30 81 B4 A0 03 02 01 11 A2 81 AC 04 81 A9 6D 55 CA 9E E9 6A 4D 9F C1 47 9A 2B 9E 75 07 73 F4 48 4A 36 AE BD 28 4D DA 3D 00 89 0D B1 70 C1 67 E7 44 0B C4 3E BF 59 3D 2B F2 EA E7 09 05 44 23 2D B6 D6 76 46 D8 DC 05 32 68 A2 2B D5 58 3D EC 5D DD 4D A1 6D F2 80 95 5F 61 2A 40 C5 D3 F7 BD A3 71 73 F2 81 DD CD B1 B3 1D 8E FA B0 70 9F 88 AD 97 7C 2A 91 DE 4F 69 5A 23 17 AA 21 99 7E 89 36 E6 1A 01 06 D2 DF 6F C3 76 15 47 81 9A 65 66 F7 CC 23 52 C5 DE 77 09 AD 66 38 94 DE 93 DC CA 24 6B C4 2B FA 1A BC FC 07 84 9B CE 0D 15 BA C7 00 8A 5C 0E 61 D8 BE 01 A4 81 98 30 81 95 A0 03 02 01 11 A2 81 8D 04 81 8A 60 DF 82 D5 14 DB 78 8D A8 E4 6D F8 FE 3A F4 AB 98 25 9D DB 51 ED 3B CE 53 C8 DC 48 1C CB EB B5 1B 5A 45 BA CD 68 0A 26 2F 8D 3A FE 75 AE 36 4B 25 B5 B8 5A C1 27 71 E3 B6 03 7D D6 2D 14 58 CD 6D 19 F0 25 D0 5A 9B 35 A6 7E 36 62 DA 28 56 0B E9 53 03 43 7B 71 D5 ED 8F 52 CE 6E 8A 23 0C 52 53 EB 42 0B 7A 6B 8C 54 EB 1C 70 FB 21 DD DF 23 B4 5E AD 42 67 65 42 61 FD DB 2D 28 C2 4D 7A 71 69 D0 74 9A 64 8A 82 A0 EC C8 A4 69 30 67 A0 07 03 05 00 00 00 00 00 A2 0D 1B 0B 45 58 41 4D 50 4C 45 2E 43 4F 4D A3 1C 30 1A A0 03 02 01 00 A1 13 30 11 1B 04 48 54 54 50 1B 09 6C 6F 63 61 6C 68 6F 73 74 A5 11 18 0F 31 39 37 30 30 31 30 31 30 30 30 30 30 30 5A A7 06 02 04 6B 88 55 A5 A8 14 30 12 02 01 12 02 01 11 02 01 10)
          	at org.apache.mina.filter.codec.ProtocolCodecFilter.messageReceived(ProtocolCodecFilter.java:234)
          	at org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:434)
          	at org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1200(DefaultIoFilterChain.java:48)
          	at org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.messageReceived(DefaultIoFilterChain.java:802)
          	at org.apache.mina.core.filterchain.IoFilterAdapter.messageReceived(IoFilterAdapter.java:120)
          	at org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:434)
          	at org.apache.mina.core.filterchain.DefaultIoFilterChain.fireMessageReceived(DefaultIoFilterChain.java:426)
          	at org.apache.mina.core.polling.AbstractPollingIoProcessor.read(AbstractPollingIoProcessor.java:604)
          	at org.apache.mina.core.polling.AbstractPollingIoProcessor.process(AbstractPollingIoProcessor.java:564)
          	at org.apache.mina.core.polling.AbstractPollingIoProcessor.process(AbstractPollingIoProcessor.java:553)
          	at org.apache.mina.core.polling.AbstractPollingIoProcessor.access$400(AbstractPollingIoProcessor.java:57)
          	at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.run(AbstractPollingIoProcessor.java:892)
          	at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:65)
          	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
          	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
          	at java.lang.Thread.run(Thread.java:745)
          Caused by: java.lang.NullPointerException: message
          	at org.apache.mina.filter.codec.AbstractProtocolDecoderOutput.write(AbstractProtocolDecoderOutput.java:44)
          	at org.apache.directory.server.kerberos.protocol.codec.MinaKerberosDecoder.decode(MinaKerberosDecoder.java:65)
          	at org.apache.mina.filter.codec.ProtocolCodecFilter.messageReceived(ProtocolCodecFilter.java:224)
          	... 15 more
          2016-02-01 06:41:48,133 ERROR server.KERBEROS_LOG (KerberosProtocolHandler.java:exceptionCaught(158)) - /127.0.0.1:46417 EXCEPTION
          org.apache.mina.filter.codec.ProtocolDecoderException: java.lang.NullPointerException: message (Hexdump: 00 00 02 45 6C 82 02 41 30 82 02 3D A1 03 02 01 05 A2 03 02 01 0C A3 82 01 C4 30 82 01 C0 30 82 01 BC A1 03 02 01 01 A2 82 01 B3 04 82 01 AF 6E 82 01 AB 30 82 01 A7 A0 03 02 01 05 A1 03 02 01 0E A2 07 03 05 00 00 00 00 00 A3 81 F6 61 81 F3 30 81 F0 A0 03 02 01 05 A1 0D 1B 0B 45 58 41 4D 50 4C 45 2E 43 4F 4D A2 20 30 1E A0 03 02 01 02 A1 17 30 15 1B 06 6B 72 62 74 67 74 1B 0B 45 58 41 4D 50 4C 45 2E 43 4F 4D A3 81 B7 30 81 B4 A0 03 02 01 11 A2 81 AC 04 81 A9 6D 55 CA 9E E9 6A 4D 9F C1 47 9A 2B 9E 75 07 73 F4 48 4A 36 AE BD 28 4D DA 3D 00 89 0D B1 70 C1 67 E7 44 0B C4 3E BF 59 3D 2B F2 EA E7 09 05 44 23 2D B6 D6 76 46 D8 DC 05 32 68 A2 2B D5 58 3D EC 5D DD 4D A1 6D F2 80 95 5F 61 2A 40 C5 D3 F7 BD A3 71 73 F2 81 DD CD B1 B3 1D 8E FA B0 70 9F 88 AD 97 7C 2A 91 DE 4F 69 5A 23 17 AA 21 99 7E 89 36 E6 1A 01 06 D2 DF 6F C3 76 15 47 81 9A 65 66 F7 CC 23 52 C5 DE 77 09 AD 66 38 94 DE 93 DC CA 24 6B C4 2B FA 1A BC FC 07 84 9B CE 0D 15 BA C7 00 8A 5C 0E 61 D8 BE 01 A4 81 98 30 81 95 A0 03 02 01 11 A2 81 8D 04 81 8A 60 DF 82 D5 14 DB 78 8D A8 E4 6D F8 FE 3A F4 AB 98 25 9D DB 51 ED 3B CE 53 C8 DC 48 1C CB EB B5 1B 5A 45 BA CD 68 0A 26 2F 8D 3A FE 75 AE 36 4B 25 B5 B8 5A C1 27 71 E3 B6 03 7D D6 2D 14 58 CD 6D 19 F0 25 D0 5A 9B 35 A6 7E 36 62 DA 28 56 0B E9 53 03 43 7B 71 D5 ED 8F 52 CE 6E 8A 23 0C 52 53 EB 42 0B 7A 6B 8C 54 EB 1C 70 FB 21 DD DF 23 B4 5E AD 42 67 65 42 61 FD DB 2D 28 C2 4D 7A 71 69 D0 74 9A 64 8A 82 A0 EC C8 A4 69 30 67 A0 07 03 05 00 00 00 00 00 A2 0D 1B 0B 45 58 41 4D 50 4C 45 2E 43 4F 4D A3 1C 30 1A A0 03 02 01 00 A1 13 30 11 1B 04 48 54 54 50 1B 09 6C 6F 63 61 6C 68 6F 73 74 A5 11 18 0F 31 39 37 30 30 31 30 31 30 30 30 30 30 30 5A A7 06 02 04 6B 88 55 A5 A8 14 30 12 02 01 12 02 01 11 02 01 10)
          	at org.apache.mina.filter.codec.ProtocolCodecFilter.messageReceived(ProtocolCodecFilter.java:234)
          	at org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:434)
          	at org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1200(DefaultIoFilterChain.java:48)
          	at org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.messageReceived(DefaultIoFilterChain.java:802)
          	at org.apache.mina.core.filterchain.IoFilterAdapter.messageReceived(IoFilterAdapter.java:120)
          	at org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:434)
          	at org.apache.mina.core.filterchain.DefaultIoFilterChain.fireMessageReceived(DefaultIoFilterChain.java:426)
          	at org.apache.mina.core.polling.AbstractPollingIoProcessor.read(AbstractPollingIoProcessor.java:604)
          	at org.apache.mina.core.polling.AbstractPollingIoProcessor.process(AbstractPollingIoProcessor.java:564)
          	at org.apache.mina.core.polling.AbstractPollingIoProcessor.process(AbstractPollingIoProcessor.java:553)
          	at org.apache.mina.core.polling.AbstractPollingIoProcessor.access$400(AbstractPollingIoProcessor.java:57)
          	at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.run(AbstractPollingIoProcessor.java:892)
          	at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:65)
          	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
          	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
          	at java.lang.Thread.run(Thread.java:745)
          Caused by: java.lang.NullPointerException: message
          	at org.apache.mina.filter.codec.AbstractProtocolDecoderOutput.write(AbstractProtocolDecoderOutput.java:44)
          	at org.apache.directory.server.kerberos.protocol.codec.MinaKerberosDecoder.decode(MinaKerberosDecoder.java:65)
          	at org.apache.mina.filter.codec.ProtocolCodecFilter.messageReceived(ProtocolCodecFilter.java:224)
          	... 15 more
          2016-02-01 06:41:48,138 ERROR protocol.KerberosProtocolHandler (KerberosProtocolHandler.java:exceptionCaught(157)) - /127.0.0.1:46418 EXCEPTION
          org.apache.mina.filter.codec.ProtocolDecoderException: java.lang.NullPointerException: message (Hexdump: 00 00 02 45 6C 82 02 41 30 82 02 3D A1 03 02 01 05 A2 03 02 01 0C A3 82 01 C4 30 82 01 C0 30 82 01 BC A1 03 02 01 01 A2 82 01 B3 04 82 01 AF 6E 82 01 AB 30 82 01 A7 A0 03 02 01 05 A1 03 02 01 0E A2 07 03 05 00 00 00 00 00 A3 81 F6 61 81 F3 30 81 F0 A0 03 02 01 05 A1 0D 1B 0B 45 58 41 4D 50 4C 45 2E 43 4F 4D A2 20 30 1E A0 03 02 01 02 A1 17 30 15 1B 06 6B 72 62 74 67 74 1B 0B 45 58 41 4D 50 4C 45 2E 43 4F 4D A3 81 B7 30 81 B4 A0 03 02 01 11 A2 81 AC 04 81 A9 6D 55 CA 9E E9 6A 4D 9F C1 47 9A 2B 9E 75 07 73 F4 48 4A 36 AE BD 28 4D DA 3D 00 89 0D B1 70 C1 67 E7 44 0B C4 3E BF 59 3D 2B F2 EA E7 09 05 44 23 2D B6 D6 76 46 D8 DC 05 32 68 A2 2B D5 58 3D EC 5D DD 4D A1 6D F2 80 95 5F 61 2A 40 C5 D3 F7 BD A3 71 73 F2 81 DD CD B1 B3 1D 8E FA B0 70 9F 88 AD 97 7C 2A 91 DE 4F 69 5A 23 17 AA 21 99 7E 89 36 E6 1A 01 06 D2 DF 6F C3 76 15 47 81 9A 65 66 F7 CC 23 52 C5 DE 77 09 AD 66 38 94 DE 93 DC CA 24 6B C4 2B FA 1A BC FC 07 84 9B CE 0D 15 BA C7 00 8A 5C 0E 61 D8 BE 01 A4 81 98 30 81 95 A0 03 02 01 11 A2 81 8D 04 81 8A 7E 36 73 CB EC 41 15 69 53 02 B8 E4 28 1A 8D 61 56 3E 30 4E 0F C6 28 7C 12 CF EB 25 1E A9 3B 28 99 68 3B CE F3 38 50 B0 DA D9 0F 25 3A 39 B4 12 39 44 A6 64 8D FA 1B 62 22 27 7E 64 D8 C4 FE D3 89 DC 82 E2 C2 21 7E 14 BE DE 89 C6 9D FD C5 3F 25 40 9A F7 2C 26 DD 16 E1 EF C5 66 A5 AD 1A 46 8E 44 9C 86 63 9B 80 BA 81 45 78 97 F4 C1 EF EF 05 68 77 2B 61 B2 DF 30 0B BF E1 94 69 44 2C 97 7D 54 52 F8 9C 88 14 56 A0 B7 A4 69 30 67 A0 07 03 05 00 00 00 00 00 A2 0D 1B 0B 45 58 41 4D 50 4C 45 2E 43 4F 4D A3 1C 30 1A A0 03 02 01 00 A1 13 30 11 1B 04 48 54 54 50 1B 09 6C 6F 63 61 6C 68 6F 73 74 A5 11 18 0F 31 39 37 30 30 31 30 31 30 30 30 30 30 30 5A A7 06 02 04 3F 0B C9 19 A8 14 30 12 02 01 12 02 01 11 02 01 10)
          	at org.apache.mina.filter.codec.ProtocolCodecFilter.messageReceived(ProtocolCodecFilter.java:234)
          	at org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:434)
          	at org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1200(DefaultIoFilterChain.java:48)
          	at org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.messageReceived(DefaultIoFilterChain.java:802)
          	at org.apache.mina.core.filterchain.IoFilterAdapter.messageReceived(IoFilterAdapter.java:120)
          	at org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:434)
          	at org.apache.mina.core.filterchain.DefaultIoFilterChain.fireMessageReceived(DefaultIoFilterChain.java:426)
          	at org.apache.mina.core.polling.AbstractPollingIoProcessor.read(AbstractPollingIoProcessor.java:604)
          	at org.apache.mina.core.polling.AbstractPollingIoProcessor.process(AbstractPollingIoProcessor.java:564)
          	at org.apache.mina.core.polling.AbstractPollingIoProcessor.process(AbstractPollingIoProcessor.java:553)
          	at org.apache.mina.core.polling.AbstractPollingIoProcessor.access$400(AbstractPollingIoProcessor.java:57)
          	at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.run(AbstractPollingIoProcessor.java:892)
          	at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:65)
          	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
          	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
          	at java.lang.Thread.run(Thread.java:745)
          Caused by: java.lang.NullPointerException: message
          	at org.apache.mina.filter.codec.AbstractProtocolDecoderOutput.write(AbstractProtocolDecoderOutput.java:44)
          	at org.apache.directory.server.kerberos.protocol.codec.MinaKerberosDecoder.decode(MinaKerberosDecoder.java:65)
          	at org.apache.mina.filter.codec.ProtocolCodecFilter.messageReceived(ProtocolCodecFilter.java:224)
          	... 15 more
          2016-02-01 06:41:48,139 ERROR server.KERBEROS_LOG (KerberosProtocolHandler.java:exceptionCaught(158)) - /127.0.0.1:46418 EXCEPTION
          org.apache.mina.filter.codec.ProtocolDecoderException: java.lang.NullPointerException: message (Hexdump: 00 00 02 45 6C 82 02 41 30 82 02 3D A1 03 02 01 05 A2 03 02 01 0C A3 82 01 C4 30 82 01 C0 30 82 01 BC A1 03 02 01 01 A2 82 01 B3 04 82 01 AF 6E 82 01 AB 30 82 01 A7 A0 03 02 01 05 A1 03 02 01 0E A2 07 03 05 00 00 00 00 00 A3 81 F6 61 81 F3 30 81 F0 A0 03 02 01 05 A1 0D 1B 0B 45 58 41 4D 50 4C 45 2E 43 4F 4D A2 20 30 1E A0 03 02 01 02 A1 17 30 15 1B 06 6B 72 62 74 67 74 1B 0B 45 58 41 4D 50 4C 45 2E 43 4F 4D A3 81 B7 30 81 B4 A0 03 02 01 11 A2 81 AC 04 81 A9 6D 55 CA 9E E9 6A 4D 9F C1 47 9A 2B 9E 75 07 73 F4 48 4A 36 AE BD 28 4D DA 3D 00 89 0D B1 70 C1 67 E7 44 0B C4 3E BF 59 3D 2B F2 EA E7 09 05 44 23 2D B6 D6 76 46 D8 DC 05 32 68 A2 2B D5 58 3D EC 5D DD 4D A1 6D F2 80 95 5F 61 2A 40 C5 D3 F7 BD A3 71 73 F2 81 DD CD B1 B3 1D 8E FA B0 70 9F 88 AD 97 7C 2A 91 DE 4F 69 5A 23 17 AA 21 99 7E 89 36 E6 1A 01 06 D2 DF 6F C3 76 15 47 81 9A 65 66 F7 CC 23 52 C5 DE 77 09 AD 66 38 94 DE 93 DC CA 24 6B C4 2B FA 1A BC FC 07 84 9B CE 0D 15 BA C7 00 8A 5C 0E 61 D8 BE 01 A4 81 98 30 81 95 A0 03 02 01 11 A2 81 8D 04 81 8A 7E 36 73 CB EC 41 15 69 53 02 B8 E4 28 1A 8D 61 56 3E 30 4E 0F C6 28 7C 12 CF EB 25 1E A9 3B 28 99 68 3B CE F3 38 50 B0 DA D9 0F 25 3A 39 B4 12 39 44 A6 64 8D FA 1B 62 22 27 7E 64 D8 C4 FE D3 89 DC 82 E2 C2 21 7E 14 BE DE 89 C6 9D FD C5 3F 25 40 9A F7 2C 26 DD 16 E1 EF C5 66 A5 AD 1A 46 8E 44 9C 86 63 9B 80 BA 81 45 78 97 F4 C1 EF EF 05 68 77 2B 61 B2 DF 30 0B BF E1 94 69 44 2C 97 7D 54 52 F8 9C 88 14 56 A0 B7 A4 69 30 67 A0 07 03 05 00 00 00 00 00 A2 0D 1B 0B 45 58 41 4D 50 4C 45 2E 43 4F 4D A3 1C 30 1A A0 03 02 01 00 A1 13 30 11 1B 04 48 54 54 50 1B 09 6C 6F 63 61 6C 68 6F 73 74 A5 11 18 0F 31 39 37 30 30 31 30 31 30 30 30 30 30 30 5A A7 06 02 04 3F 0B C9 19 A8 14 30 12 02 01 12 02 01 11 02 01 10)
          	at org.apache.mina.filter.codec.ProtocolCodecFilter.messageReceived(ProtocolCodecFilter.java:234)
          	at org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:434)
          	at org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1200(DefaultIoFilterChain.java:48)
          	at org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.messageReceived(DefaultIoFilterChain.java:802)
          	at org.apache.mina.core.filterchain.IoFilterAdapter.messageReceived(IoFilterAdapter.java:120)
          	at org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:434)
          	at org.apache.mina.core.filterchain.DefaultIoFilterChain.fireMessageReceived(DefaultIoFilterChain.java:426)
          	at org.apache.mina.core.polling.AbstractPollingIoProcessor.read(AbstractPollingIoProcessor.java:604)
          	at org.apache.mina.core.polling.AbstractPollingIoProcessor.process(AbstractPollingIoProcessor.java:564)
          	at org.apache.mina.core.polling.AbstractPollingIoProcessor.process(AbstractPollingIoProcessor.java:553)
          	at org.apache.mina.core.polling.AbstractPollingIoProcessor.access$400(AbstractPollingIoProcessor.java:57)
          	at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.run(AbstractPollingIoProcessor.java:892)
          	at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:65)
          	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
          	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
          	at java.lang.Thread.run(Thread.java:745)
          Caused by: java.lang.NullPointerException: message
          	at org.apache.mina.filter.codec.AbstractProtocolDecoderOutput.write(AbstractProtocolDecoderOutput.java:44)
          	at org.apache.directory.server.kerberos.protocol.codec.MinaKerberosDecoder.decode(MinaKerberosDecoder.java:65)
          	at org.apache.mina.filter.codec.ProtocolCodecFilter.messageReceived(ProtocolCodecFilter.java:224)
          	... 15 more
          2016-02-01 06:41:48,139 WARN  security.UserGroupInformation (UserGroupInformation.java:doAs(1696)) - PriviledgedActionException as:slave/localhost@EXAMPLE.COM (auth:KERBEROS) cause:java.io.IOException: org.apache.hadoop.security.authentication.client.AuthenticationException: GSSException: No valid credentials provided (Mechanism level: Cannot get a KDC reply)
          2016-02-01 06:41:48,140 ERROR namenode.EditLogInputStream (EditLogFileInputStream.java:nextOpImpl(194)) - caught exception initializing https://localhost:55326/getJournal?jid=myjournal&segmentTxId=1&storageInfo=-60%3A1226055383%3A0%3AtestClusterID
          java.io.IOException: org.apache.hadoop.security.authentication.client.AuthenticationException: GSSException: No valid credentials provided (Mechanism level: Cannot get a KDC reply)
          	at org.apache.hadoop.hdfs.server.namenode.EditLogFileInputStream$URLLog$1.run(EditLogFileInputStream.java:473)
          	at org.apache.hadoop.hdfs.server.namenode.EditLogFileInputStream$URLLog$1.run(EditLogFileInputStream.java:465)
          	at java.security.AccessController.doPrivileged(Native Method)
          	at javax.security.auth.Subject.doAs(Subject.java:415)
          	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693)
          	at org.apache.hadoop.security.SecurityUtil.doAsUser(SecurityUtil.java:445)
          	at org.apache.hadoop.security.SecurityUtil.doAsCurrentUser(SecurityUtil.java:439)
          	at org.apache.hadoop.hdfs.server.namenode.EditLogFileInputStream$URLLog.getInputStream(EditLogFileInputStream.java:464)
          	at org.apache.hadoop.hdfs.server.namenode.EditLogFileInputStream.init(EditLogFileInputStream.java:141)
          	at org.apache.hadoop.hdfs.server.namenode.EditLogFileInputStream.nextOpImpl(EditLogFileInputStream.java:192)
          	at org.apache.hadoop.hdfs.server.namenode.EditLogFileInputStream.nextOp(EditLogFileInputStream.java:250)
          	at org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.readOp(EditLogInputStream.java:85)
          	at org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.skipUntil(EditLogInputStream.java:151)
          	at org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream.nextOp(RedundantEditLogInputStream.java:178)
          	at org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.readOp(EditLogInputStream.java:85)
          	at org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.skipUntil(EditLogInputStream.java:151)
          	at org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream.nextOp(RedundantEditLogInputStream.java:178)
          	at org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.readOp(EditLogInputStream.java:85)
          	at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadEditRecords(FSEditLogLoader.java:187)
          	at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadFSEdits(FSEditLogLoader.java:140)
          	at org.apache.hadoop.hdfs.server.namenode.FSImage.loadEdits(FSImage.java:835)
          	at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:690)
          	at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:281)
          	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:1063)
          	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:767)
          	at org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:609)
          	at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:670)
          	at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:838)
          	at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:817)
          	at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1538)
          	at org.apache.hadoop.hdfs.MiniDFSCluster.restartNameNode(MiniDFSCluster.java:1862)
          	at org.apache.hadoop.hdfs.MiniDFSCluster.restartNameNode(MiniDFSCluster.java:1827)
          	at org.apache.hadoop.hdfs.qjournal.TestSecureNNWithQJM.restartNameNode(TestSecureNNWithQJM.java:197)
          	at org.apache.hadoop.hdfs.qjournal.TestSecureNNWithQJM.doNNWithQJMTest(TestSecureNNWithQJM.java:179)
          	at org.apache.hadoop.hdfs.qjournal.TestSecureNNWithQJM.testSecureMode(TestSecureNNWithQJM.java:159)
          	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
          	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
          	at java.lang.reflect.Method.invoke(Method.java:606)
          	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
          	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
          	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
          	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
          	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
          	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
          	at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
          Caused by: org.apache.hadoop.security.authentication.client.AuthenticationException: GSSException: No valid credentials provided (Mechanism level: Cannot get a KDC reply)
          	at org.apache.hadoop.security.authentication.client.KerberosAuthenticator.doSpnegoSequence(KerberosAuthenticator.java:306)
          	at org.apache.hadoop.security.authentication.client.KerberosAuthenticator.authenticate(KerberosAuthenticator.java:196)
          	at org.apache.hadoop.security.authentication.client.AuthenticatedURL.openConnection(AuthenticatedURL.java:215)
          	at org.apache.hadoop.hdfs.web.URLConnectionFactory.openConnection(URLConnectionFactory.java:161)
          	at org.apache.hadoop.hdfs.server.namenode.EditLogFileInputStream$URLLog$1.run(EditLogFileInputStream.java:470)
          	... 45 more
          Caused by: GSSException: No valid credentials provided (Mechanism level: Cannot get a KDC reply)
          	at sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:715)
          	at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:248)
          	at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
          	at org.apache.hadoop.security.authentication.client.KerberosAuthenticator$1.run(KerberosAuthenticator.java:285)
          	at org.apache.hadoop.security.authentication.client.KerberosAuthenticator$1.run(KerberosAuthenticator.java:261)
          	at java.security.AccessController.doPrivileged(Native Method)
          	at javax.security.auth.Subject.doAs(Subject.java:415)
          	at org.apache.hadoop.security.authentication.client.KerberosAuthenticator.doSpnegoSequence(KerberosAuthenticator.java:261)
          	... 49 more
          Caused by: java.io.IOException: Cannot get a KDC reply
          	at sun.security.krb5.KdcComm.send(KdcComm.java:250)
          	at sun.security.krb5.KdcComm.send(KdcComm.java:191)
          	at sun.security.krb5.KrbTgsReq.send(KrbTgsReq.java:186)
          	at sun.security.krb5.KrbTgsReq.sendAndGetCreds(KrbTgsReq.java:201)
          	at sun.security.krb5.internal.CredentialsUtil.serviceCreds(CredentialsUtil.java:292)
          	at sun.security.krb5.internal.CredentialsUtil.acquireServiceCreds(CredentialsUtil.java:101)
          	at sun.security.krb5.Credentials.acquireServiceCreds(Credentials.java:456)
          	at sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:641)
          	... 56 more
          2016-02-01 06:41:48,141 ERROR namenode.EditLogInputStream (RedundantEditLogInputStream.java:nextOp(221)) - Got error reading edit log input stream https://localhost:55326/getJournal?jid=myjournal&segmentTxId=1&storageInfo=-60%3A1226055383%3A0%3AtestClusterID; failing over to edit log https://localhost:48398/getJournal?jid=myjournal&segmentTxId=1&storageInfo=-60%3A1226055383%3A0%3AtestClusterID
          org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream$PrematureEOFException: got premature end-of-file at txid 0; expected file to go up to 5
          	at org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream.nextOp(RedundantEditLogInputStream.java:194)
          	at org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.readOp(EditLogInputStream.java:85)
          	at org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.skipUntil(EditLogInputStream.java:151)
          	at org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream.nextOp(RedundantEditLogInputStream.java:178)
          	at org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.readOp(EditLogInputStream.java:85)
          	at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadEditRecords(FSEditLogLoader.java:187)
          	at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadFSEdits(FSEditLogLoader.java:140)
          	at org.apache.hadoop.hdfs.server.namenode.FSImage.loadEdits(FSImage.java:835)
          	at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:690)
          	at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:281)
          	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:1063)
          	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:767)
          	at org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:609)
          	at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:670)
          	at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:838)
          	at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:817)
          	at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1538)
          	at org.apache.hadoop.hdfs.MiniDFSCluster.restartNameNode(MiniDFSCluster.java:1862)
          	at org.apache.hadoop.hdfs.MiniDFSCluster.restartNameNode(MiniDFSCluster.java:1827)
          	at org.apache.hadoop.hdfs.qjournal.TestSecureNNWithQJM.restartNameNode(TestSecureNNWithQJM.java:197)
          	at org.apache.hadoop.hdfs.qjournal.TestSecureNNWithQJM.doNNWithQJMTest(TestSecureNNWithQJM.java:179)
          	at org.apache.hadoop.hdfs.qjournal.TestSecureNNWithQJM.testSecureMode(TestSecureNNWithQJM.java:159)
          	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
          	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
          	at java.lang.reflect.Method.invoke(Method.java:606)
          	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
          	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
          	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
          	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
          	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
          	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
          	at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
          2016-02-01 06:41:48,142 INFO  namenode.EditLogInputStream (RedundantEditLogInputStream.java:nextOp(176)) - Fast-forwarding stream 'https://localhost:48398/getJournal?jid=myjournal&segmentTxId=1&storageInfo=-60%3A1226055383%3A0%3AtestClusterID' to transaction ID 1
          2016-02-01 06:41:48,169 ERROR protocol.KerberosProtocolHandler (KerberosProtocolHandler.java:exceptionCaught(157)) - /127.0.0.1:46420 EXCEPTION
          org.apache.mina.filter.codec.ProtocolDecoderException: java.lang.NullPointerException: message (Hexdump: 00 00 02 45 6C 82 02 41 30 82 02 3D A1 03 02 01 05 A2 03 02 01 0C A3 82 01 C4 30 82 01 C0 30 82 01 BC A1 03 02 01 01 A2 82 01 B3 04 82 01 AF 6E 82 01 AB 30 82 01 A7 A0 03 02 01 05 A1 03 02 01 0E A2 07 03 05 00 00 00 00 00 A3 81 F6 61 81 F3 30 81 F0 A0 03 02 01 05 A1 0D 1B 0B 45 58 41 4D 50 4C 45 2E 43 4F 4D A2 20 30 1E A0 03 02 01 02 A1 17 30 15 1B 06 6B 72 62 74 67 74 1B 0B 45 58 41 4D 50 4C 45 2E 43 4F 4D A3 81 B7 30 81 B4 A0 03 02 01 11 A2 81 AC 04 81 A9 6D 55 CA 9E E9 6A 4D 9F C1 47 9A 2B 9E 75 07 73 F4 48 4A 36 AE BD 28 4D DA 3D 00 89 0D B1 70 C1 67 E7 44 0B C4 3E BF 59 3D 2B F2 EA E7 09 05 44 23 2D B6 D6 76 46 D8 DC 05 32 68 A2 2B D5 58 3D EC 5D DD 4D A1 6D F2 80 95 5F 61 2A 40 C5 D3 F7 BD A3 71 73 F2 81 DD CD B1 B3 1D 8E FA B0 70 9F 88 AD 97 7C 2A 91 DE 4F 69 5A 23 17 AA 21 99 7E 89 36 E6 1A 01 06 D2 DF 6F C3 76 15 47 81 9A 65 66 F7 CC 23 52 C5 DE 77 09 AD 66 38 94 DE 93 DC CA 24 6B C4 2B FA 1A BC FC 07 84 9B CE 0D 15 BA C7 00 8A 5C 0E 61 D8 BE 01 A4 81 98 30 81 95 A0 03 02 01 11 A2 81 8D 04 81 8A BA 11 25 9F 01 D1 21 8A 31 30 5B 14 7A 6D A5 65 C1 87 95 F1 86 3A E1 A9 7E 3E 7F 4D FE 8E 8F 96 E4 0A 78 2B 5F 0D 1D 87 88 51 A4 5B 92 94 6A FC 87 97 2A 9C 69 81 B3 5D 5D 67 56 E9 97 97 5D 29 7B B9 B8 9E 99 8F 7A BD 3A E3 4D 6E BA 55 0C ED 3A 67 0F A9 45 B5 93 9B 48 FF FB 45 A7 23 5A B9 52 44 E4 54 D5 08 D5 2F FF 4A C4 52 16 FA 5F 1C 62 41 88 15 C1 B6 63 13 DA 16 33 56 63 37 09 63 2E 1D C8 42 55 FB FA 33 9B CC A4 69 30 67 A0 07 03 05 00 00 00 00 00 A2 0D 1B 0B 45 58 41 4D 50 4C 45 2E 43 4F 4D A3 1C 30 1A A0 03 02 01 00 A1 13 30 11 1B 04 48 54 54 50 1B 09 6C 6F 63 61 6C 68 6F 73 74 A5 11 18 0F 31 39 37 30 30 31 30 31 30 30 30 30 30 30 5A A7 06 02 04 01 AA D4 AF A8 14 30 12 02 01 12 02 01 11 02 01 10)
          	at org.apache.mina.filter.codec.ProtocolCodecFilter.messageReceived(ProtocolCodecFilter.java:234)
          	at org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:434)
          	at org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1200(DefaultIoFilterChain.java:48)
          	at org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.messageReceived(DefaultIoFilterChain.java:802)
          	at org.apache.mina.core.filterchain.IoFilterAdapter.messageReceived(IoFilterAdapter.java:120)
          	at org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:434)
          	at org.apache.mina.core.filterchain.DefaultIoFilterChain.fireMessageReceived(DefaultIoFilterChain.java:426)
          	at org.apache.mina.core.polling.AbstractPollingIoProcessor.read(AbstractPollingIoProcessor.java:604)
          	at org.apache.mina.core.polling.AbstractPollingIoProcessor.process(AbstractPollingIoProcessor.java:564)
          	at org.apache.mina.core.polling.AbstractPollingIoProcessor.process(AbstractPollingIoProcessor.java:553)
          	at org.apache.mina.core.polling.AbstractPollingIoProcessor.access$400(AbstractPollingIoProcessor.java:57)
          	at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.run(AbstractPollingIoProcessor.java:892)
          	at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:65)
          	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
          	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
          	at java.lang.Thread.run(Thread.java:745)
          Caused by: java.lang.NullPointerException: message
          	at org.apache.mina.filter.codec.AbstractProtocolDecoderOutput.write(AbstractProtocolDecoderOutput.java:44)
          	at org.apache.directory.server.kerberos.protocol.codec.MinaKerberosDecoder.decode(MinaKerberosDecoder.java:65)
          	at org.apache.mina.filter.codec.ProtocolCodecFilter.messageReceived(ProtocolCodecFilter.java:224)
          	... 15 more
          2016-02-01 06:41:48,169 ERROR server.KERBEROS_LOG (KerberosProtocolHandler.java:exceptionCaught(158)) - /127.0.0.1:46420 EXCEPTION
          org.apache.mina.filter.codec.ProtocolDecoderException: java.lang.NullPointerException: message (Hexdump: 00 00 02 45 6C 82 02 41 30 82 02 3D A1 03 02 01 05 A2 03 02 01 0C A3 82 01 C4 30 82 01 C0 30 82 01 BC A1 03 02 01 01 A2 82 01 B3 04 82 01 AF 6E 82 01 AB 30 82 01 A7 A0 03 02 01 05 A1 03 02 01 0E A2 07 03 05 00 00 00 00 00 A3 81 F6 61 81 F3 30 81 F0 A0 03 02 01 05 A1 0D 1B 0B 45 58 41 4D 50 4C 45 2E 43 4F 4D A2 20 30 1E A0 03 02 01 02 A1 17 30 15 1B 06 6B 72 62 74 67 74 1B 0B 45 58 41 4D 50 4C 45 2E 43 4F 4D A3 81 B7 30 81 B4 A0 03 02 01 11 A2 81 AC 04 81 A9 6D 55 CA 9E E9 6A 4D 9F C1 47 9A 2B 9E 75 07 73 F4 48 4A 36 AE BD 28 4D DA 3D 00 89 0D B1 70 C1 67 E7 44 0B C4 3E BF 59 3D 2B F2 EA E7 09 05 44 23 2D B6 D6 76 46 D8 DC 05 32 68 A2 2B D5 58 3D EC 5D DD 4D A1 6D F2 80 95 5F 61 2A 40 C5 D3 F7 BD A3 71 73 F2 81 DD CD B1 B3 1D 8E FA B0 70 9F 88 AD 97 7C 2A 91 DE 4F 69 5A 23 17 AA 21 99 7E 89 36 E6 1A 01 06 D2 DF 6F C3 76 15 47 81 9A 65 66 F7 CC 23 52 C5 DE 77 09 AD 66 38 94 DE 93 DC CA 24 6B C4 2B FA 1A BC FC 07 84 9B CE 0D 15 BA C7 00 8A 5C 0E 61 D8 BE 01 A4 81 98 30 81 95 A0 03 02 01 11 A2 81 8D 04 81 8A BA 11 25 9F 01 D1 21 8A 31 30 5B 14 7A 6D A5 65 C1 87 95 F1 86 3A E1 A9 7E 3E 7F 4D FE 8E 8F 96 E4 0A 78 2B 5F 0D 1D 87 88 51 A4 5B 92 94 6A FC 87 97 2A 9C 69 81 B3 5D 5D 67 56 E9 97 97 5D 29 7B B9 B8 9E 99 8F 7A BD 3A E3 4D 6E BA 55 0C ED 3A 67 0F A9 45 B5 93 9B 48 FF FB 45 A7 23 5A B9 52 44 E4 54 D5 08 D5 2F FF 4A C4 52 16 FA 5F 1C 62 41 88 15 C1 B6 63 13 DA 16 33 56 63 37 09 63 2E 1D C8 42 55 FB FA 33 9B CC A4 69 30 67 A0 07 03 05 00 00 00 00 00 A2 0D 1B 0B 45 58 41 4D 50 4C 45 2E 43 4F 4D A3 1C 30 1A A0 03 02 01 00 A1 13 30 11 1B 04 48 54 54 50 1B 09 6C 6F 63 61 6C 68 6F 73 74 A5 11 18 0F 31 39 37 30 30 31 30 31 30 30 30 30 30 30 5A A7 06 02 04 01 AA D4 AF A8 14 30 12 02 01 12 02 01 11 02 01 10)
          	at org.apache.mina.filter.codec.ProtocolCodecFilter.messageReceived(ProtocolCodecFilter.java:234)
          	at org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:434)
          	at org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1200(DefaultIoFilterChain.java:48)
          	at org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.messageReceived(DefaultIoFilterChain.java:802)
          	at org.apache.mina.core.filterchain.IoFilterAdapter.messageReceived(IoFilterAdapter.java:120)
          	at org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:434)
          	at org.apache.mina.core.filterchain.DefaultIoFilterChain.fireMessageReceived(DefaultIoFilterChain.java:426)
          	at org.apache.mina.core.polling.AbstractPollingIoProcessor.read(AbstractPollingIoProcessor.java:604)
          	at org.apache.mina.core.polling.AbstractPollingIoProcessor.process(AbstractPollingIoProcessor.java:564)
          	at org.apache.mina.core.polling.AbstractPollingIoProcessor.process(AbstractPollingIoProcessor.java:553)
          	at org.apache.mina.core.polling.AbstractPollingIoProcessor.access$400(AbstractPollingIoProcessor.java:57)
          	at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.run(AbstractPollingIoProcessor.java:892)
          	at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:65)
          	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
          	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
          	at java.lang.Thread.run(Thread.java:745)
          Caused by: java.lang.NullPointerException: message
          	at org.apache.mina.filter.codec.AbstractProtocolDecoderOutput.write(AbstractProtocolDecoderOutput.java:44)
          	at org.apache.directory.server.kerberos.protocol.codec.MinaKerberosDecoder.decode(MinaKerberosDecoder.java:65)
          	at org.apache.mina.filter.codec.ProtocolCodecFilter.messageReceived(ProtocolCodecFilter.java:224)
          	... 15 more
          2016-02-01 06:41:48,170 WARN  security.UserGroupInformation (UserGroupInformation.java:doAs(1696)) - PriviledgedActionException as:slave/localhost@EXAMPLE.COM (auth:KERBEROS) cause:java.io.IOException: org.apache.hadoop.security.authentication.client.AuthenticationException: GSSException: No valid credentials provided (Mechanism level: Cannot get a KDC reply)
          2016-02-01 06:41:48,170 ERROR namenode.EditLogInputStream (EditLogFileInputStream.java:nextOpImpl(194)) - caught exception initializing https://localhost:48398/getJournal?jid=myjournal&segmentTxId=1&storageInfo=-60%3A1226055383%3A0%3AtestClusterID
          java.io.IOException: org.apache.hadoop.security.authentication.client.AuthenticationException: GSSException: No valid credentials provided (Mechanism level: Cannot get a KDC reply)
          	at org.apache.hadoop.hdfs.server.namenode.EditLogFileInputStream$URLLog$1.run(EditLogFileInputStream.java:473)
          	at org.apache.hadoop.hdfs.server.namenode.EditLogFileInputStream$URLLog$1.run(EditLogFileInputStream.java:465)
          	at java.security.AccessController.doPrivileged(Native Method)
          	at javax.security.auth.Subject.doAs(Subject.java:415)
          	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693)
          	at org.apache.hadoop.security.SecurityUtil.doAsUser(SecurityUtil.java:445)
          	at org.apache.hadoop.security.SecurityUtil.doAsCurrentUser(SecurityUtil.java:439)
          	at org.apache.hadoop.hdfs.server.namenode.EditLogFileInputStream$URLLog.getInputStream(EditLogFileInputStream.java:464)
          	at org.apache.hadoop.hdfs.server.namenode.EditLogFileInputStream.init(EditLogFileInputStream.java:141)
          	at org.apache.hadoop.hdfs.server.namenode.EditLogFileInputStream.nextOpImpl(EditLogFileInputStream.java:192)
          	at org.apache.hadoop.hdfs.server.namenode.EditLogFileInputStream.nextOp(EditLogFileInputStream.java:250)
          	at org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.readOp(EditLogInputStream.java:85)
          	at org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.skipUntil(EditLogInputStream.java:151)
          	at org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream.nextOp(RedundantEditLogInputStream.java:178)
          	at org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.readOp(EditLogInputStream.java:85)
          	at org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.skipUntil(EditLogInputStream.java:151)
          	at org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream.nextOp(RedundantEditLogInputStream.java:178)
          	at org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.readOp(EditLogInputStream.java:85)
          	at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadEditRecords(FSEditLogLoader.java:187)
          	at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadFSEdits(FSEditLogLoader.java:140)
          	at org.apache.hadoop.hdfs.server.namenode.FSImage.loadEdits(FSImage.java:835)
          	at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:690)
          	at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:281)
          	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:1063)
          	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:767)
          	at org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:609)
          	at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:670)
          	at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:838)
          	at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:817)
          	at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1538)
          	at org.apache.hadoop.hdfs.MiniDFSCluster.restartNameNode(MiniDFSCluster.java:1862)
          	at org.apache.hadoop.hdfs.MiniDFSCluster.restartNameNode(MiniDFSCluster.java:1827)
          	at org.apache.hadoop.hdfs.qjournal.TestSecureNNWithQJM.restartNameNode(TestSecureNNWithQJM.java:197)
          	at org.apache.hadoop.hdfs.qjournal.TestSecureNNWithQJM.doNNWithQJMTest(TestSecureNNWithQJM.java:179)
          	at org.apache.hadoop.hdfs.qjournal.TestSecureNNWithQJM.testSecureMode(TestSecureNNWithQJM.java:159)
          	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
          	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
          	at java.lang.reflect.Method.invoke(Method.java:606)
          	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
          	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
          	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
          	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
          	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
          	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
          	at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
          Caused by: org.apache.hadoop.security.authentication.client.AuthenticationException: GSSException: No valid credentials provided (Mechanism level: Cannot get a KDC reply)
          	at org.apache.hadoop.security.authentication.client.KerberosAuthenticator.doSpnegoSequence(KerberosAuthenticator.java:306)
          	at org.apache.hadoop.security.authentication.client.KerberosAuthenticator.authenticate(KerberosAuthenticator.java:196)
          	at org.apache.hadoop.security.authentication.client.AuthenticatedURL.openConnection(AuthenticatedURL.java:215)
          	at org.apache.hadoop.hdfs.web.URLConnectionFactory.openConnection(URLConnectionFactory.java:161)
          	at org.apache.hadoop.hdfs.server.namenode.EditLogFileInputStream$URLLog$1.run(EditLogFileInputStream.java:470)
          	... 45 more
          Caused by: GSSException: No valid credentials provided (Mechanism level: Cannot get a KDC reply)
          	at sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:715)
          	at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:248)
          	at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
          	at org.apache.hadoop.security.authentication.client.KerberosAuthenticator$1.run(KerberosAuthenticator.java:285)
          	at org.apache.hadoop.security.authentication.client.KerberosAuthenticator$1.run(KerberosAuthenticator.java:261)
          	at java.security.AccessController.doPrivileged(Native Method)
          	at javax.security.auth.Subject.doAs(Subject.java:415)
          	at org.apache.hadoop.security.authentication.client.KerberosAuthenticator.doSpnegoSequence(KerberosAuthenticator.java:261)
          	... 49 more
          Caused by: java.io.IOException: Cannot get a KDC reply
          	at sun.security.krb5.KdcComm.send(KdcComm.java:250)
          	at sun.security.krb5.KdcComm.send(KdcComm.java:191)
          	at sun.security.krb5.KrbTgsReq.send(KrbTgsReq.java:186)
          	at sun.security.krb5.KrbTgsReq.sendAndGetCreds(KrbTgsReq.java:201)
          	at sun.security.krb5.internal.CredentialsUtil.serviceCreds(CredentialsUtil.java:292)
          	at sun.security.krb5.internal.CredentialsUtil.acquireServiceCreds(CredentialsUtil.java:101)
          	at sun.security.krb5.Credentials.acquireServiceCreds(Credentials.java:456)
          	at sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:641)
          	... 56 more
          2016-02-01 06:41:48,171 ERROR namenode.FSImage (FSEditLogLoader.java:loadEditRecords(196)) - Error replaying edit log at offset 0.  Expected transaction ID was 1
          org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream$PrematureEOFException: got premature end-of-file at txid 0; expected file to go up to 5
          	at org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream.nextOp(RedundantEditLogInputStream.java:194)
          	at org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.readOp(EditLogInputStream.java:85)
          	at org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.skipUntil(EditLogInputStream.java:151)
          	at org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream.nextOp(RedundantEditLogInputStream.java:178)
          	at org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.readOp(EditLogInputStream.java:85)
          	at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadEditRecords(FSEditLogLoader.java:187)
          	at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadFSEdits(FSEditLogLoader.java:140)
          	at org.apache.hadoop.hdfs.server.namenode.FSImage.loadEdits(FSImage.java:835)
          	at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:690)
          	at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:281)
          	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:1063)
          	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:767)
          	at org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:609)
          	at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:670)
          	at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:838)
          	at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:817)
          	at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1538)
          	at org.apache.hadoop.hdfs.MiniDFSCluster.restartNameNode(MiniDFSCluster.java:1862)
          	at org.apache.hadoop.hdfs.MiniDFSCluster.restartNameNode(MiniDFSCluster.java:1827)
          	at org.apache.hadoop.hdfs.qjournal.TestSecureNNWithQJM.restartNameNode(TestSecureNNWithQJM.java:197)
          	at org.apache.hadoop.hdfs.qjournal.TestSecureNNWithQJM.doNNWithQJMTest(TestSecureNNWithQJM.java:179)
          	at org.apache.hadoop.hdfs.qjournal.TestSecureNNWithQJM.testSecureMode(TestSecureNNWithQJM.java:159)
          	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
          	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
          	at java.lang.reflect.Method.invoke(Method.java:606)
          	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
          	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
          	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
          	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
          	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
          	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
          	at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
          2016-02-01 06:41:48,173 WARN  namenode.FSNamesystem (FSNamesystem.java:loadFromDisk(769)) - Encountered exception loading fsimage
          org.apache.hadoop.hdfs.server.namenode.EditLogInputException: Error replaying edit log at offset 0.  Expected transaction ID was 1
          	at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadEditRecords(FSEditLogLoader.java:200)
          	at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadFSEdits(FSEditLogLoader.java:140)
          	at org.apache.hadoop.hdfs.server.namenode.FSImage.loadEdits(FSImage.java:835)
          	at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:690)
          	at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:281)
          	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:1063)
          	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:767)
          	at org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:609)
          	at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:670)
          	at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:838)
          	at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:817)
          	at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1538)
          	at org.apache.hadoop.hdfs.MiniDFSCluster.restartNameNode(MiniDFSCluster.java:1862)
          	at org.apache.hadoop.hdfs.MiniDFSCluster.restartNameNode(MiniDFSCluster.java:1827)
          	at org.apache.hadoop.hdfs.qjournal.TestSecureNNWithQJM.restartNameNode(TestSecureNNWithQJM.java:197)
          	at org.apache.hadoop.hdfs.qjournal.TestSecureNNWithQJM.doNNWithQJMTest(TestSecureNNWithQJM.java:179)
          	at org.apache.hadoop.hdfs.qjournal.TestSecureNNWithQJM.testSecureMode(TestSecureNNWithQJM.java:159)
          	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
          	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
          	at java.lang.reflect.Method.invoke(Method.java:606)
          	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
          	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
          	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
          	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
          	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
          	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
          	at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
          Caused by: org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream$PrematureEOFException: got premature end-of-file at txid 0; expected file to go up to 5
          	at org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream.nextOp(RedundantEditLogInputStream.java:194)
          	at org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.readOp(EditLogInputStream.java:85)
          	at org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.skipUntil(EditLogInputStream.java:151)
          	at org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream.nextOp(RedundantEditLogInputStream.java:178)
          	at org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.readOp(EditLogInputStream.java:85)
          	at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadEditRecords(FSEditLogLoader.java:187)
          	... 27 more
          2016-02-01 06:41:48,187 INFO  mortbay.log (Slf4jLog.java:info(67)) - Stopped SslSocketConnectorSecure@localhost:41570
          2016-02-01 06:41:48,287 INFO  hdfs.MiniDFSCluster (MiniDFSCluster.java:shutdown(1744)) - Shutting down the Mini HDFS Cluster
          
          Show
          xiaochen Xiao Chen added a comment - Here is the more sophisticated error. I haven't root caused it, but kerberos seems to be involved. I've also seen this NPE in TestKMS#testACLs failures.. 2016-02-01 06:41:47,944 INFO namenode.FSImage (FSImage.java:loadEdits(832)) - Reading org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream@12e8c783 expecting start txid #1 2016-02-01 06:41:47,944 INFO namenode.FSImage (FSEditLogLoader.java:loadFSEdits(139)) - Start loading edits file https://localhost:55326/getJournal?jid=myjournal&segmentTxId=1&storageInfo=-60%3A1226055383%3A0%3AtestClusterID, https://localhost:48398/getJournal?jid=myjournal&segmentTxId=1&storageInfo=-60%3A1226055383%3A0%3AtestClusterID 2016-02-01 06:41:47,945 INFO namenode.EditLogInputStream (RedundantEditLogInputStream.java:nextOp(176)) - Fast-forwarding stream 'https://localhost:55326/getJournal?jid=myjournal&segmentTxId=1&storageInfo=-60%3A1226055383%3A0%3AtestClusterID, https://localhost:48398/getJournal?jid=myjournal&segmentTxId=1&storageInfo=-60%3A1226055383%3A0%3AtestClusterID' to transaction ID 1 2016-02-01 06:41:47,945 INFO namenode.EditLogInputStream (RedundantEditLogInputStream.java:nextOp(176)) - Fast-forwarding stream 'https://localhost:55326/getJournal?jid=myjournal&segmentTxId=1&storageInfo=-60%3A1226055383%3A0%3AtestClusterID' to transaction ID 1 2016-02-01 06:41:48,131 ERROR protocol.KerberosProtocolHandler (KerberosProtocolHandler.java:exceptionCaught(157)) - /127.0.0.1:46417 EXCEPTION org.apache.mina.filter.codec.ProtocolDecoderException: java.lang.NullPointerException: message (Hexdump: 00 00 02 45 6C 82 02 41 30 82 02 3D A1 03 02 01 05 A2 03 02 01 0C A3 82 01 C4 30 82 01 C0 30 82 01 BC A1 03 02 01 01 A2 82 01 B3 04 82 01 AF 6E 82 01 AB 30 82 01 A7 A0 03 02 01 05 A1 03 02 01 0E A2 07 03 05 00 00 00 00 00 A3 81 F6 61 81 F3 30 81 F0 A0 03 02 01 05 A1 0D 1B 0B 45 58 41 4D 50 4C 45 2E 43 4F 4D A2 20 30 1E A0 03 02 01 02 A1 17 30 15 1B 06 6B 72 62 74 67 74 1B 0B 45 58 41 4D 50 4C 45 2E 43 4F 4D A3 81 B7 30 81 B4 A0 03 02 01 11 A2 81 AC 04 81 A9 6D 55 CA 9E E9 6A 4D 9F C1 47 9A 2B 9E 75 07 73 F4 48 4A 36 AE BD 28 4D DA 3D 00 89 0D B1 70 C1 67 E7 44 0B C4 3E BF 59 3D 2B F2 EA E7 09 05 44 23 2D B6 D6 76 46 D8 DC 05 32 68 A2 2B D5 58 3D EC 5D DD 4D A1 6D F2 80 95 5F 61 2A 40 C5 D3 F7 BD A3 71 73 F2 81 DD CD B1 B3 1D 8E FA B0 70 9F 88 AD 97 7C 2A 91 DE 4F 69 5A 23 17 AA 21 99 7E 89 36 E6 1A 01 06 D2 DF 6F C3 76 15 47 81 9A 65 66 F7 CC 23 52 C5 DE 77 09 AD 66 38 94 DE 93 DC CA 24 6B C4 2B FA 1A BC FC 07 84 9B CE 0D 15 BA C7 00 8A 5C 0E 61 D8 BE 01 A4 81 98 30 81 95 A0 03 02 01 11 A2 81 8D 04 81 8A 60 DF 82 D5 14 DB 78 8D A8 E4 6D F8 FE 3A F4 AB 98 25 9D DB 51 ED 3B CE 53 C8 DC 48 1C CB EB B5 1B 5A 45 BA CD 68 0A 26 2F 8D 3A FE 75 AE 36 4B 25 B5 B8 5A C1 27 71 E3 B6 03 7D D6 2D 14 58 CD 6D 19 F0 25 D0 5A 9B 35 A6 7E 36 62 DA 28 56 0B E9 53 03 43 7B 71 D5 ED 8F 52 CE 6E 8A 23 0C 52 53 EB 42 0B 7A 6B 8C 54 EB 1C 70 FB 21 DD DF 23 B4 5E AD 42 67 65 42 61 FD DB 2D 28 C2 4D 7A 71 69 D0 74 9A 64 8A 82 A0 EC C8 A4 69 30 67 A0 07 03 05 00 00 00 00 00 A2 0D 1B 0B 45 58 41 4D 50 4C 45 2E 43 4F 4D A3 1C 30 1A A0 03 02 01 00 A1 13 30 11 1B 04 48 54 54 50 1B 09 6C 6F 63 61 6C 68 6F 73 74 A5 11 18 0F 31 39 37 30 30 31 30 31 30 30 30 30 30 30 5A A7 06 02 04 6B 88 55 A5 A8 14 30 12 02 01 12 02 01 11 02 01 10) at org.apache.mina.filter.codec.ProtocolCodecFilter.messageReceived(ProtocolCodecFilter.java:234) at org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:434) at org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1200(DefaultIoFilterChain.java:48) at org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.messageReceived(DefaultIoFilterChain.java:802) at org.apache.mina.core.filterchain.IoFilterAdapter.messageReceived(IoFilterAdapter.java:120) at org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:434) at org.apache.mina.core.filterchain.DefaultIoFilterChain.fireMessageReceived(DefaultIoFilterChain.java:426) at org.apache.mina.core.polling.AbstractPollingIoProcessor.read(AbstractPollingIoProcessor.java:604) at org.apache.mina.core.polling.AbstractPollingIoProcessor.process(AbstractPollingIoProcessor.java:564) at org.apache.mina.core.polling.AbstractPollingIoProcessor.process(AbstractPollingIoProcessor.java:553) at org.apache.mina.core.polling.AbstractPollingIoProcessor.access$400(AbstractPollingIoProcessor.java:57) at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.run(AbstractPollingIoProcessor.java:892) at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:65) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.NullPointerException: message at org.apache.mina.filter.codec.AbstractProtocolDecoderOutput.write(AbstractProtocolDecoderOutput.java:44) at org.apache.directory.server.kerberos.protocol.codec.MinaKerberosDecoder.decode(MinaKerberosDecoder.java:65) at org.apache.mina.filter.codec.ProtocolCodecFilter.messageReceived(ProtocolCodecFilter.java:224) ... 15 more 2016-02-01 06:41:48,133 ERROR server.KERBEROS_LOG (KerberosProtocolHandler.java:exceptionCaught(158)) - /127.0.0.1:46417 EXCEPTION org.apache.mina.filter.codec.ProtocolDecoderException: java.lang.NullPointerException: message (Hexdump: 00 00 02 45 6C 82 02 41 30 82 02 3D A1 03 02 01 05 A2 03 02 01 0C A3 82 01 C4 30 82 01 C0 30 82 01 BC A1 03 02 01 01 A2 82 01 B3 04 82 01 AF 6E 82 01 AB 30 82 01 A7 A0 03 02 01 05 A1 03 02 01 0E A2 07 03 05 00 00 00 00 00 A3 81 F6 61 81 F3 30 81 F0 A0 03 02 01 05 A1 0D 1B 0B 45 58 41 4D 50 4C 45 2E 43 4F 4D A2 20 30 1E A0 03 02 01 02 A1 17 30 15 1B 06 6B 72 62 74 67 74 1B 0B 45 58 41 4D 50 4C 45 2E 43 4F 4D A3 81 B7 30 81 B4 A0 03 02 01 11 A2 81 AC 04 81 A9 6D 55 CA 9E E9 6A 4D 9F C1 47 9A 2B 9E 75 07 73 F4 48 4A 36 AE BD 28 4D DA 3D 00 89 0D B1 70 C1 67 E7 44 0B C4 3E BF 59 3D 2B F2 EA E7 09 05 44 23 2D B6 D6 76 46 D8 DC 05 32 68 A2 2B D5 58 3D EC 5D DD 4D A1 6D F2 80 95 5F 61 2A 40 C5 D3 F7 BD A3 71 73 F2 81 DD CD B1 B3 1D 8E FA B0 70 9F 88 AD 97 7C 2A 91 DE 4F 69 5A 23 17 AA 21 99 7E 89 36 E6 1A 01 06 D2 DF 6F C3 76 15 47 81 9A 65 66 F7 CC 23 52 C5 DE 77 09 AD 66 38 94 DE 93 DC CA 24 6B C4 2B FA 1A BC FC 07 84 9B CE 0D 15 BA C7 00 8A 5C 0E 61 D8 BE 01 A4 81 98 30 81 95 A0 03 02 01 11 A2 81 8D 04 81 8A 60 DF 82 D5 14 DB 78 8D A8 E4 6D F8 FE 3A F4 AB 98 25 9D DB 51 ED 3B CE 53 C8 DC 48 1C CB EB B5 1B 5A 45 BA CD 68 0A 26 2F 8D 3A FE 75 AE 36 4B 25 B5 B8 5A C1 27 71 E3 B6 03 7D D6 2D 14 58 CD 6D 19 F0 25 D0 5A 9B 35 A6 7E 36 62 DA 28 56 0B E9 53 03 43 7B 71 D5 ED 8F 52 CE 6E 8A 23 0C 52 53 EB 42 0B 7A 6B 8C 54 EB 1C 70 FB 21 DD DF 23 B4 5E AD 42 67 65 42 61 FD DB 2D 28 C2 4D 7A 71 69 D0 74 9A 64 8A 82 A0 EC C8 A4 69 30 67 A0 07 03 05 00 00 00 00 00 A2 0D 1B 0B 45 58 41 4D 50 4C 45 2E 43 4F 4D A3 1C 30 1A A0 03 02 01 00 A1 13 30 11 1B 04 48 54 54 50 1B 09 6C 6F 63 61 6C 68 6F 73 74 A5 11 18 0F 31 39 37 30 30 31 30 31 30 30 30 30 30 30 5A A7 06 02 04 6B 88 55 A5 A8 14 30 12 02 01 12 02 01 11 02 01 10) at org.apache.mina.filter.codec.ProtocolCodecFilter.messageReceived(ProtocolCodecFilter.java:234) at org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:434) at org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1200(DefaultIoFilterChain.java:48) at org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.messageReceived(DefaultIoFilterChain.java:802) at org.apache.mina.core.filterchain.IoFilterAdapter.messageReceived(IoFilterAdapter.java:120) at org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:434) at org.apache.mina.core.filterchain.DefaultIoFilterChain.fireMessageReceived(DefaultIoFilterChain.java:426) at org.apache.mina.core.polling.AbstractPollingIoProcessor.read(AbstractPollingIoProcessor.java:604) at org.apache.mina.core.polling.AbstractPollingIoProcessor.process(AbstractPollingIoProcessor.java:564) at org.apache.mina.core.polling.AbstractPollingIoProcessor.process(AbstractPollingIoProcessor.java:553) at org.apache.mina.core.polling.AbstractPollingIoProcessor.access$400(AbstractPollingIoProcessor.java:57) at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.run(AbstractPollingIoProcessor.java:892) at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:65) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.NullPointerException: message at org.apache.mina.filter.codec.AbstractProtocolDecoderOutput.write(AbstractProtocolDecoderOutput.java:44) at org.apache.directory.server.kerberos.protocol.codec.MinaKerberosDecoder.decode(MinaKerberosDecoder.java:65) at org.apache.mina.filter.codec.ProtocolCodecFilter.messageReceived(ProtocolCodecFilter.java:224) ... 15 more 2016-02-01 06:41:48,138 ERROR protocol.KerberosProtocolHandler (KerberosProtocolHandler.java:exceptionCaught(157)) - /127.0.0.1:46418 EXCEPTION org.apache.mina.filter.codec.ProtocolDecoderException: java.lang.NullPointerException: message (Hexdump: 00 00 02 45 6C 82 02 41 30 82 02 3D A1 03 02 01 05 A2 03 02 01 0C A3 82 01 C4 30 82 01 C0 30 82 01 BC A1 03 02 01 01 A2 82 01 B3 04 82 01 AF 6E 82 01 AB 30 82 01 A7 A0 03 02 01 05 A1 03 02 01 0E A2 07 03 05 00 00 00 00 00 A3 81 F6 61 81 F3 30 81 F0 A0 03 02 01 05 A1 0D 1B 0B 45 58 41 4D 50 4C 45 2E 43 4F 4D A2 20 30 1E A0 03 02 01 02 A1 17 30 15 1B 06 6B 72 62 74 67 74 1B 0B 45 58 41 4D 50 4C 45 2E 43 4F 4D A3 81 B7 30 81 B4 A0 03 02 01 11 A2 81 AC 04 81 A9 6D 55 CA 9E E9 6A 4D 9F C1 47 9A 2B 9E 75 07 73 F4 48 4A 36 AE BD 28 4D DA 3D 00 89 0D B1 70 C1 67 E7 44 0B C4 3E BF 59 3D 2B F2 EA E7 09 05 44 23 2D B6 D6 76 46 D8 DC 05 32 68 A2 2B D5 58 3D EC 5D DD 4D A1 6D F2 80 95 5F 61 2A 40 C5 D3 F7 BD A3 71 73 F2 81 DD CD B1 B3 1D 8E FA B0 70 9F 88 AD 97 7C 2A 91 DE 4F 69 5A 23 17 AA 21 99 7E 89 36 E6 1A 01 06 D2 DF 6F C3 76 15 47 81 9A 65 66 F7 CC 23 52 C5 DE 77 09 AD 66 38 94 DE 93 DC CA 24 6B C4 2B FA 1A BC FC 07 84 9B CE 0D 15 BA C7 00 8A 5C 0E 61 D8 BE 01 A4 81 98 30 81 95 A0 03 02 01 11 A2 81 8D 04 81 8A 7E 36 73 CB EC 41 15 69 53 02 B8 E4 28 1A 8D 61 56 3E 30 4E 0F C6 28 7C 12 CF EB 25 1E A9 3B 28 99 68 3B CE F3 38 50 B0 DA D9 0F 25 3A 39 B4 12 39 44 A6 64 8D FA 1B 62 22 27 7E 64 D8 C4 FE D3 89 DC 82 E2 C2 21 7E 14 BE DE 89 C6 9D FD C5 3F 25 40 9A F7 2C 26 DD 16 E1 EF C5 66 A5 AD 1A 46 8E 44 9C 86 63 9B 80 BA 81 45 78 97 F4 C1 EF EF 05 68 77 2B 61 B2 DF 30 0B BF E1 94 69 44 2C 97 7D 54 52 F8 9C 88 14 56 A0 B7 A4 69 30 67 A0 07 03 05 00 00 00 00 00 A2 0D 1B 0B 45 58 41 4D 50 4C 45 2E 43 4F 4D A3 1C 30 1A A0 03 02 01 00 A1 13 30 11 1B 04 48 54 54 50 1B 09 6C 6F 63 61 6C 68 6F 73 74 A5 11 18 0F 31 39 37 30 30 31 30 31 30 30 30 30 30 30 5A A7 06 02 04 3F 0B C9 19 A8 14 30 12 02 01 12 02 01 11 02 01 10) at org.apache.mina.filter.codec.ProtocolCodecFilter.messageReceived(ProtocolCodecFilter.java:234) at org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:434) at org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1200(DefaultIoFilterChain.java:48) at org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.messageReceived(DefaultIoFilterChain.java:802) at org.apache.mina.core.filterchain.IoFilterAdapter.messageReceived(IoFilterAdapter.java:120) at org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:434) at org.apache.mina.core.filterchain.DefaultIoFilterChain.fireMessageReceived(DefaultIoFilterChain.java:426) at org.apache.mina.core.polling.AbstractPollingIoProcessor.read(AbstractPollingIoProcessor.java:604) at org.apache.mina.core.polling.AbstractPollingIoProcessor.process(AbstractPollingIoProcessor.java:564) at org.apache.mina.core.polling.AbstractPollingIoProcessor.process(AbstractPollingIoProcessor.java:553) at org.apache.mina.core.polling.AbstractPollingIoProcessor.access$400(AbstractPollingIoProcessor.java:57) at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.run(AbstractPollingIoProcessor.java:892) at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:65) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.NullPointerException: message at org.apache.mina.filter.codec.AbstractProtocolDecoderOutput.write(AbstractProtocolDecoderOutput.java:44) at org.apache.directory.server.kerberos.protocol.codec.MinaKerberosDecoder.decode(MinaKerberosDecoder.java:65) at org.apache.mina.filter.codec.ProtocolCodecFilter.messageReceived(ProtocolCodecFilter.java:224) ... 15 more 2016-02-01 06:41:48,139 ERROR server.KERBEROS_LOG (KerberosProtocolHandler.java:exceptionCaught(158)) - /127.0.0.1:46418 EXCEPTION org.apache.mina.filter.codec.ProtocolDecoderException: java.lang.NullPointerException: message (Hexdump: 00 00 02 45 6C 82 02 41 30 82 02 3D A1 03 02 01 05 A2 03 02 01 0C A3 82 01 C4 30 82 01 C0 30 82 01 BC A1 03 02 01 01 A2 82 01 B3 04 82 01 AF 6E 82 01 AB 30 82 01 A7 A0 03 02 01 05 A1 03 02 01 0E A2 07 03 05 00 00 00 00 00 A3 81 F6 61 81 F3 30 81 F0 A0 03 02 01 05 A1 0D 1B 0B 45 58 41 4D 50 4C 45 2E 43 4F 4D A2 20 30 1E A0 03 02 01 02 A1 17 30 15 1B 06 6B 72 62 74 67 74 1B 0B 45 58 41 4D 50 4C 45 2E 43 4F 4D A3 81 B7 30 81 B4 A0 03 02 01 11 A2 81 AC 04 81 A9 6D 55 CA 9E E9 6A 4D 9F C1 47 9A 2B 9E 75 07 73 F4 48 4A 36 AE BD 28 4D DA 3D 00 89 0D B1 70 C1 67 E7 44 0B C4 3E BF 59 3D 2B F2 EA E7 09 05 44 23 2D B6 D6 76 46 D8 DC 05 32 68 A2 2B D5 58 3D EC 5D DD 4D A1 6D F2 80 95 5F 61 2A 40 C5 D3 F7 BD A3 71 73 F2 81 DD CD B1 B3 1D 8E FA B0 70 9F 88 AD 97 7C 2A 91 DE 4F 69 5A 23 17 AA 21 99 7E 89 36 E6 1A 01 06 D2 DF 6F C3 76 15 47 81 9A 65 66 F7 CC 23 52 C5 DE 77 09 AD 66 38 94 DE 93 DC CA 24 6B C4 2B FA 1A BC FC 07 84 9B CE 0D 15 BA C7 00 8A 5C 0E 61 D8 BE 01 A4 81 98 30 81 95 A0 03 02 01 11 A2 81 8D 04 81 8A 7E 36 73 CB EC 41 15 69 53 02 B8 E4 28 1A 8D 61 56 3E 30 4E 0F C6 28 7C 12 CF EB 25 1E A9 3B 28 99 68 3B CE F3 38 50 B0 DA D9 0F 25 3A 39 B4 12 39 44 A6 64 8D FA 1B 62 22 27 7E 64 D8 C4 FE D3 89 DC 82 E2 C2 21 7E 14 BE DE 89 C6 9D FD C5 3F 25 40 9A F7 2C 26 DD 16 E1 EF C5 66 A5 AD 1A 46 8E 44 9C 86 63 9B 80 BA 81 45 78 97 F4 C1 EF EF 05 68 77 2B 61 B2 DF 30 0B BF E1 94 69 44 2C 97 7D 54 52 F8 9C 88 14 56 A0 B7 A4 69 30 67 A0 07 03 05 00 00 00 00 00 A2 0D 1B 0B 45 58 41 4D 50 4C 45 2E 43 4F 4D A3 1C 30 1A A0 03 02 01 00 A1 13 30 11 1B 04 48 54 54 50 1B 09 6C 6F 63 61 6C 68 6F 73 74 A5 11 18 0F 31 39 37 30 30 31 30 31 30 30 30 30 30 30 5A A7 06 02 04 3F 0B C9 19 A8 14 30 12 02 01 12 02 01 11 02 01 10) at org.apache.mina.filter.codec.ProtocolCodecFilter.messageReceived(ProtocolCodecFilter.java:234) at org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:434) at org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1200(DefaultIoFilterChain.java:48) at org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.messageReceived(DefaultIoFilterChain.java:802) at org.apache.mina.core.filterchain.IoFilterAdapter.messageReceived(IoFilterAdapter.java:120) at org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:434) at org.apache.mina.core.filterchain.DefaultIoFilterChain.fireMessageReceived(DefaultIoFilterChain.java:426) at org.apache.mina.core.polling.AbstractPollingIoProcessor.read(AbstractPollingIoProcessor.java:604) at org.apache.mina.core.polling.AbstractPollingIoProcessor.process(AbstractPollingIoProcessor.java:564) at org.apache.mina.core.polling.AbstractPollingIoProcessor.process(AbstractPollingIoProcessor.java:553) at org.apache.mina.core.polling.AbstractPollingIoProcessor.access$400(AbstractPollingIoProcessor.java:57) at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.run(AbstractPollingIoProcessor.java:892) at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:65) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.NullPointerException: message at org.apache.mina.filter.codec.AbstractProtocolDecoderOutput.write(AbstractProtocolDecoderOutput.java:44) at org.apache.directory.server.kerberos.protocol.codec.MinaKerberosDecoder.decode(MinaKerberosDecoder.java:65) at org.apache.mina.filter.codec.ProtocolCodecFilter.messageReceived(ProtocolCodecFilter.java:224) ... 15 more 2016-02-01 06:41:48,139 WARN security.UserGroupInformation (UserGroupInformation.java:doAs(1696)) - PriviledgedActionException as:slave/localhost@EXAMPLE.COM (auth:KERBEROS) cause:java.io.IOException: org.apache.hadoop.security.authentication.client.AuthenticationException: GSSException: No valid credentials provided (Mechanism level: Cannot get a KDC reply) 2016-02-01 06:41:48,140 ERROR namenode.EditLogInputStream (EditLogFileInputStream.java:nextOpImpl(194)) - caught exception initializing https://localhost:55326/getJournal?jid=myjournal&segmentTxId=1&storageInfo=-60%3A1226055383%3A0%3AtestClusterID java.io.IOException: org.apache.hadoop.security.authentication.client.AuthenticationException: GSSException: No valid credentials provided (Mechanism level: Cannot get a KDC reply) at org.apache.hadoop.hdfs.server.namenode.EditLogFileInputStream$URLLog$1.run(EditLogFileInputStream.java:473) at org.apache.hadoop.hdfs.server.namenode.EditLogFileInputStream$URLLog$1.run(EditLogFileInputStream.java:465) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693) at org.apache.hadoop.security.SecurityUtil.doAsUser(SecurityUtil.java:445) at org.apache.hadoop.security.SecurityUtil.doAsCurrentUser(SecurityUtil.java:439) at org.apache.hadoop.hdfs.server.namenode.EditLogFileInputStream$URLLog.getInputStream(EditLogFileInputStream.java:464) at org.apache.hadoop.hdfs.server.namenode.EditLogFileInputStream.init(EditLogFileInputStream.java:141) at org.apache.hadoop.hdfs.server.namenode.EditLogFileInputStream.nextOpImpl(EditLogFileInputStream.java:192) at org.apache.hadoop.hdfs.server.namenode.EditLogFileInputStream.nextOp(EditLogFileInputStream.java:250) at org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.readOp(EditLogInputStream.java:85) at org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.skipUntil(EditLogInputStream.java:151) at org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream.nextOp(RedundantEditLogInputStream.java:178) at org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.readOp(EditLogInputStream.java:85) at org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.skipUntil(EditLogInputStream.java:151) at org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream.nextOp(RedundantEditLogInputStream.java:178) at org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.readOp(EditLogInputStream.java:85) at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadEditRecords(FSEditLogLoader.java:187) at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadFSEdits(FSEditLogLoader.java:140) at org.apache.hadoop.hdfs.server.namenode.FSImage.loadEdits(FSImage.java:835) at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:690) at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:281) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:1063) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:767) at org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:609) at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:670) at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:838) at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:817) at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1538) at org.apache.hadoop.hdfs.MiniDFSCluster.restartNameNode(MiniDFSCluster.java:1862) at org.apache.hadoop.hdfs.MiniDFSCluster.restartNameNode(MiniDFSCluster.java:1827) at org.apache.hadoop.hdfs.qjournal.TestSecureNNWithQJM.restartNameNode(TestSecureNNWithQJM.java:197) at org.apache.hadoop.hdfs.qjournal.TestSecureNNWithQJM.doNNWithQJMTest(TestSecureNNWithQJM.java:179) at org.apache.hadoop.hdfs.qjournal.TestSecureNNWithQJM.testSecureMode(TestSecureNNWithQJM.java:159) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74) Caused by: org.apache.hadoop.security.authentication.client.AuthenticationException: GSSException: No valid credentials provided (Mechanism level: Cannot get a KDC reply) at org.apache.hadoop.security.authentication.client.KerberosAuthenticator.doSpnegoSequence(KerberosAuthenticator.java:306) at org.apache.hadoop.security.authentication.client.KerberosAuthenticator.authenticate(KerberosAuthenticator.java:196) at org.apache.hadoop.security.authentication.client.AuthenticatedURL.openConnection(AuthenticatedURL.java:215) at org.apache.hadoop.hdfs.web.URLConnectionFactory.openConnection(URLConnectionFactory.java:161) at org.apache.hadoop.hdfs.server.namenode.EditLogFileInputStream$URLLog$1.run(EditLogFileInputStream.java:470) ... 45 more Caused by: GSSException: No valid credentials provided (Mechanism level: Cannot get a KDC reply) at sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:715) at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:248) at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179) at org.apache.hadoop.security.authentication.client.KerberosAuthenticator$1.run(KerberosAuthenticator.java:285) at org.apache.hadoop.security.authentication.client.KerberosAuthenticator$1.run(KerberosAuthenticator.java:261) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.authentication.client.KerberosAuthenticator.doSpnegoSequence(KerberosAuthenticator.java:261) ... 49 more Caused by: java.io.IOException: Cannot get a KDC reply at sun.security.krb5.KdcComm.send(KdcComm.java:250) at sun.security.krb5.KdcComm.send(KdcComm.java:191) at sun.security.krb5.KrbTgsReq.send(KrbTgsReq.java:186) at sun.security.krb5.KrbTgsReq.sendAndGetCreds(KrbTgsReq.java:201) at sun.security.krb5.internal.CredentialsUtil.serviceCreds(CredentialsUtil.java:292) at sun.security.krb5.internal.CredentialsUtil.acquireServiceCreds(CredentialsUtil.java:101) at sun.security.krb5.Credentials.acquireServiceCreds(Credentials.java:456) at sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:641) ... 56 more 2016-02-01 06:41:48,141 ERROR namenode.EditLogInputStream (RedundantEditLogInputStream.java:nextOp(221)) - Got error reading edit log input stream https://localhost:55326/getJournal?jid=myjournal&segmentTxId=1&storageInfo=-60%3A1226055383%3A0%3AtestClusterID; failing over to edit log https://localhost:48398/getJournal?jid=myjournal&segmentTxId=1&storageInfo=-60%3A1226055383%3A0%3AtestClusterID org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream$PrematureEOFException: got premature end-of-file at txid 0; expected file to go up to 5 at org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream.nextOp(RedundantEditLogInputStream.java:194) at org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.readOp(EditLogInputStream.java:85) at org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.skipUntil(EditLogInputStream.java:151) at org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream.nextOp(RedundantEditLogInputStream.java:178) at org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.readOp(EditLogInputStream.java:85) at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadEditRecords(FSEditLogLoader.java:187) at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadFSEdits(FSEditLogLoader.java:140) at org.apache.hadoop.hdfs.server.namenode.FSImage.loadEdits(FSImage.java:835) at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:690) at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:281) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:1063) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:767) at org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:609) at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:670) at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:838) at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:817) at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1538) at org.apache.hadoop.hdfs.MiniDFSCluster.restartNameNode(MiniDFSCluster.java:1862) at org.apache.hadoop.hdfs.MiniDFSCluster.restartNameNode(MiniDFSCluster.java:1827) at org.apache.hadoop.hdfs.qjournal.TestSecureNNWithQJM.restartNameNode(TestSecureNNWithQJM.java:197) at org.apache.hadoop.hdfs.qjournal.TestSecureNNWithQJM.doNNWithQJMTest(TestSecureNNWithQJM.java:179) at org.apache.hadoop.hdfs.qjournal.TestSecureNNWithQJM.testSecureMode(TestSecureNNWithQJM.java:159) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74) 2016-02-01 06:41:48,142 INFO namenode.EditLogInputStream (RedundantEditLogInputStream.java:nextOp(176)) - Fast-forwarding stream 'https://localhost:48398/getJournal?jid=myjournal&segmentTxId=1&storageInfo=-60%3A1226055383%3A0%3AtestClusterID' to transaction ID 1 2016-02-01 06:41:48,169 ERROR protocol.KerberosProtocolHandler (KerberosProtocolHandler.java:exceptionCaught(157)) - /127.0.0.1:46420 EXCEPTION org.apache.mina.filter.codec.ProtocolDecoderException: java.lang.NullPointerException: message (Hexdump: 00 00 02 45 6C 82 02 41 30 82 02 3D A1 03 02 01 05 A2 03 02 01 0C A3 82 01 C4 30 82 01 C0 30 82 01 BC A1 03 02 01 01 A2 82 01 B3 04 82 01 AF 6E 82 01 AB 30 82 01 A7 A0 03 02 01 05 A1 03 02 01 0E A2 07 03 05 00 00 00 00 00 A3 81 F6 61 81 F3 30 81 F0 A0 03 02 01 05 A1 0D 1B 0B 45 58 41 4D 50 4C 45 2E 43 4F 4D A2 20 30 1E A0 03 02 01 02 A1 17 30 15 1B 06 6B 72 62 74 67 74 1B 0B 45 58 41 4D 50 4C 45 2E 43 4F 4D A3 81 B7 30 81 B4 A0 03 02 01 11 A2 81 AC 04 81 A9 6D 55 CA 9E E9 6A 4D 9F C1 47 9A 2B 9E 75 07 73 F4 48 4A 36 AE BD 28 4D DA 3D 00 89 0D B1 70 C1 67 E7 44 0B C4 3E BF 59 3D 2B F2 EA E7 09 05 44 23 2D B6 D6 76 46 D8 DC 05 32 68 A2 2B D5 58 3D EC 5D DD 4D A1 6D F2 80 95 5F 61 2A 40 C5 D3 F7 BD A3 71 73 F2 81 DD CD B1 B3 1D 8E FA B0 70 9F 88 AD 97 7C 2A 91 DE 4F 69 5A 23 17 AA 21 99 7E 89 36 E6 1A 01 06 D2 DF 6F C3 76 15 47 81 9A 65 66 F7 CC 23 52 C5 DE 77 09 AD 66 38 94 DE 93 DC CA 24 6B C4 2B FA 1A BC FC 07 84 9B CE 0D 15 BA C7 00 8A 5C 0E 61 D8 BE 01 A4 81 98 30 81 95 A0 03 02 01 11 A2 81 8D 04 81 8A BA 11 25 9F 01 D1 21 8A 31 30 5B 14 7A 6D A5 65 C1 87 95 F1 86 3A E1 A9 7E 3E 7F 4D FE 8E 8F 96 E4 0A 78 2B 5F 0D 1D 87 88 51 A4 5B 92 94 6A FC 87 97 2A 9C 69 81 B3 5D 5D 67 56 E9 97 97 5D 29 7B B9 B8 9E 99 8F 7A BD 3A E3 4D 6E BA 55 0C ED 3A 67 0F A9 45 B5 93 9B 48 FF FB 45 A7 23 5A B9 52 44 E4 54 D5 08 D5 2F FF 4A C4 52 16 FA 5F 1C 62 41 88 15 C1 B6 63 13 DA 16 33 56 63 37 09 63 2E 1D C8 42 55 FB FA 33 9B CC A4 69 30 67 A0 07 03 05 00 00 00 00 00 A2 0D 1B 0B 45 58 41 4D 50 4C 45 2E 43 4F 4D A3 1C 30 1A A0 03 02 01 00 A1 13 30 11 1B 04 48 54 54 50 1B 09 6C 6F 63 61 6C 68 6F 73 74 A5 11 18 0F 31 39 37 30 30 31 30 31 30 30 30 30 30 30 5A A7 06 02 04 01 AA D4 AF A8 14 30 12 02 01 12 02 01 11 02 01 10) at org.apache.mina.filter.codec.ProtocolCodecFilter.messageReceived(ProtocolCodecFilter.java:234) at org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:434) at org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1200(DefaultIoFilterChain.java:48) at org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.messageReceived(DefaultIoFilterChain.java:802) at org.apache.mina.core.filterchain.IoFilterAdapter.messageReceived(IoFilterAdapter.java:120) at org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:434) at org.apache.mina.core.filterchain.DefaultIoFilterChain.fireMessageReceived(DefaultIoFilterChain.java:426) at org.apache.mina.core.polling.AbstractPollingIoProcessor.read(AbstractPollingIoProcessor.java:604) at org.apache.mina.core.polling.AbstractPollingIoProcessor.process(AbstractPollingIoProcessor.java:564) at org.apache.mina.core.polling.AbstractPollingIoProcessor.process(AbstractPollingIoProcessor.java:553) at org.apache.mina.core.polling.AbstractPollingIoProcessor.access$400(AbstractPollingIoProcessor.java:57) at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.run(AbstractPollingIoProcessor.java:892) at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:65) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.NullPointerException: message at org.apache.mina.filter.codec.AbstractProtocolDecoderOutput.write(AbstractProtocolDecoderOutput.java:44) at org.apache.directory.server.kerberos.protocol.codec.MinaKerberosDecoder.decode(MinaKerberosDecoder.java:65) at org.apache.mina.filter.codec.ProtocolCodecFilter.messageReceived(ProtocolCodecFilter.java:224) ... 15 more 2016-02-01 06:41:48,169 ERROR server.KERBEROS_LOG (KerberosProtocolHandler.java:exceptionCaught(158)) - /127.0.0.1:46420 EXCEPTION org.apache.mina.filter.codec.ProtocolDecoderException: java.lang.NullPointerException: message (Hexdump: 00 00 02 45 6C 82 02 41 30 82 02 3D A1 03 02 01 05 A2 03 02 01 0C A3 82 01 C4 30 82 01 C0 30 82 01 BC A1 03 02 01 01 A2 82 01 B3 04 82 01 AF 6E 82 01 AB 30 82 01 A7 A0 03 02 01 05 A1 03 02 01 0E A2 07 03 05 00 00 00 00 00 A3 81 F6 61 81 F3 30 81 F0 A0 03 02 01 05 A1 0D 1B 0B 45 58 41 4D 50 4C 45 2E 43 4F 4D A2 20 30 1E A0 03 02 01 02 A1 17 30 15 1B 06 6B 72 62 74 67 74 1B 0B 45 58 41 4D 50 4C 45 2E 43 4F 4D A3 81 B7 30 81 B4 A0 03 02 01 11 A2 81 AC 04 81 A9 6D 55 CA 9E E9 6A 4D 9F C1 47 9A 2B 9E 75 07 73 F4 48 4A 36 AE BD 28 4D DA 3D 00 89 0D B1 70 C1 67 E7 44 0B C4 3E BF 59 3D 2B F2 EA E7 09 05 44 23 2D B6 D6 76 46 D8 DC 05 32 68 A2 2B D5 58 3D EC 5D DD 4D A1 6D F2 80 95 5F 61 2A 40 C5 D3 F7 BD A3 71 73 F2 81 DD CD B1 B3 1D 8E FA B0 70 9F 88 AD 97 7C 2A 91 DE 4F 69 5A 23 17 AA 21 99 7E 89 36 E6 1A 01 06 D2 DF 6F C3 76 15 47 81 9A 65 66 F7 CC 23 52 C5 DE 77 09 AD 66 38 94 DE 93 DC CA 24 6B C4 2B FA 1A BC FC 07 84 9B CE 0D 15 BA C7 00 8A 5C 0E 61 D8 BE 01 A4 81 98 30 81 95 A0 03 02 01 11 A2 81 8D 04 81 8A BA 11 25 9F 01 D1 21 8A 31 30 5B 14 7A 6D A5 65 C1 87 95 F1 86 3A E1 A9 7E 3E 7F 4D FE 8E 8F 96 E4 0A 78 2B 5F 0D 1D 87 88 51 A4 5B 92 94 6A FC 87 97 2A 9C 69 81 B3 5D 5D 67 56 E9 97 97 5D 29 7B B9 B8 9E 99 8F 7A BD 3A E3 4D 6E BA 55 0C ED 3A 67 0F A9 45 B5 93 9B 48 FF FB 45 A7 23 5A B9 52 44 E4 54 D5 08 D5 2F FF 4A C4 52 16 FA 5F 1C 62 41 88 15 C1 B6 63 13 DA 16 33 56 63 37 09 63 2E 1D C8 42 55 FB FA 33 9B CC A4 69 30 67 A0 07 03 05 00 00 00 00 00 A2 0D 1B 0B 45 58 41 4D 50 4C 45 2E 43 4F 4D A3 1C 30 1A A0 03 02 01 00 A1 13 30 11 1B 04 48 54 54 50 1B 09 6C 6F 63 61 6C 68 6F 73 74 A5 11 18 0F 31 39 37 30 30 31 30 31 30 30 30 30 30 30 5A A7 06 02 04 01 AA D4 AF A8 14 30 12 02 01 12 02 01 11 02 01 10) at org.apache.mina.filter.codec.ProtocolCodecFilter.messageReceived(ProtocolCodecFilter.java:234) at org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:434) at org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1200(DefaultIoFilterChain.java:48) at org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.messageReceived(DefaultIoFilterChain.java:802) at org.apache.mina.core.filterchain.IoFilterAdapter.messageReceived(IoFilterAdapter.java:120) at org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:434) at org.apache.mina.core.filterchain.DefaultIoFilterChain.fireMessageReceived(DefaultIoFilterChain.java:426) at org.apache.mina.core.polling.AbstractPollingIoProcessor.read(AbstractPollingIoProcessor.java:604) at org.apache.mina.core.polling.AbstractPollingIoProcessor.process(AbstractPollingIoProcessor.java:564) at org.apache.mina.core.polling.AbstractPollingIoProcessor.process(AbstractPollingIoProcessor.java:553) at org.apache.mina.core.polling.AbstractPollingIoProcessor.access$400(AbstractPollingIoProcessor.java:57) at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.run(AbstractPollingIoProcessor.java:892) at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:65) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.NullPointerException: message at org.apache.mina.filter.codec.AbstractProtocolDecoderOutput.write(AbstractProtocolDecoderOutput.java:44) at org.apache.directory.server.kerberos.protocol.codec.MinaKerberosDecoder.decode(MinaKerberosDecoder.java:65) at org.apache.mina.filter.codec.ProtocolCodecFilter.messageReceived(ProtocolCodecFilter.java:224) ... 15 more 2016-02-01 06:41:48,170 WARN security.UserGroupInformation (UserGroupInformation.java:doAs(1696)) - PriviledgedActionException as:slave/localhost@EXAMPLE.COM (auth:KERBEROS) cause:java.io.IOException: org.apache.hadoop.security.authentication.client.AuthenticationException: GSSException: No valid credentials provided (Mechanism level: Cannot get a KDC reply) 2016-02-01 06:41:48,170 ERROR namenode.EditLogInputStream (EditLogFileInputStream.java:nextOpImpl(194)) - caught exception initializing https://localhost:48398/getJournal?jid=myjournal&segmentTxId=1&storageInfo=-60%3A1226055383%3A0%3AtestClusterID java.io.IOException: org.apache.hadoop.security.authentication.client.AuthenticationException: GSSException: No valid credentials provided (Mechanism level: Cannot get a KDC reply) at org.apache.hadoop.hdfs.server.namenode.EditLogFileInputStream$URLLog$1.run(EditLogFileInputStream.java:473) at org.apache.hadoop.hdfs.server.namenode.EditLogFileInputStream$URLLog$1.run(EditLogFileInputStream.java:465) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693) at org.apache.hadoop.security.SecurityUtil.doAsUser(SecurityUtil.java:445) at org.apache.hadoop.security.SecurityUtil.doAsCurrentUser(SecurityUtil.java:439) at org.apache.hadoop.hdfs.server.namenode.EditLogFileInputStream$URLLog.getInputStream(EditLogFileInputStream.java:464) at org.apache.hadoop.hdfs.server.namenode.EditLogFileInputStream.init(EditLogFileInputStream.java:141) at org.apache.hadoop.hdfs.server.namenode.EditLogFileInputStream.nextOpImpl(EditLogFileInputStream.java:192) at org.apache.hadoop.hdfs.server.namenode.EditLogFileInputStream.nextOp(EditLogFileInputStream.java:250) at org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.readOp(EditLogInputStream.java:85) at org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.skipUntil(EditLogInputStream.java:151) at org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream.nextOp(RedundantEditLogInputStream.java:178) at org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.readOp(EditLogInputStream.java:85) at org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.skipUntil(EditLogInputStream.java:151) at org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream.nextOp(RedundantEditLogInputStream.java:178) at org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.readOp(EditLogInputStream.java:85) at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadEditRecords(FSEditLogLoader.java:187) at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadFSEdits(FSEditLogLoader.java:140) at org.apache.hadoop.hdfs.server.namenode.FSImage.loadEdits(FSImage.java:835) at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:690) at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:281) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:1063) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:767) at org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:609) at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:670) at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:838) at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:817) at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1538) at org.apache.hadoop.hdfs.MiniDFSCluster.restartNameNode(MiniDFSCluster.java:1862) at org.apache.hadoop.hdfs.MiniDFSCluster.restartNameNode(MiniDFSCluster.java:1827) at org.apache.hadoop.hdfs.qjournal.TestSecureNNWithQJM.restartNameNode(TestSecureNNWithQJM.java:197) at org.apache.hadoop.hdfs.qjournal.TestSecureNNWithQJM.doNNWithQJMTest(TestSecureNNWithQJM.java:179) at org.apache.hadoop.hdfs.qjournal.TestSecureNNWithQJM.testSecureMode(TestSecureNNWithQJM.java:159) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74) Caused by: org.apache.hadoop.security.authentication.client.AuthenticationException: GSSException: No valid credentials provided (Mechanism level: Cannot get a KDC reply) at org.apache.hadoop.security.authentication.client.KerberosAuthenticator.doSpnegoSequence(KerberosAuthenticator.java:306) at org.apache.hadoop.security.authentication.client.KerberosAuthenticator.authenticate(KerberosAuthenticator.java:196) at org.apache.hadoop.security.authentication.client.AuthenticatedURL.openConnection(AuthenticatedURL.java:215) at org.apache.hadoop.hdfs.web.URLConnectionFactory.openConnection(URLConnectionFactory.java:161) at org.apache.hadoop.hdfs.server.namenode.EditLogFileInputStream$URLLog$1.run(EditLogFileInputStream.java:470) ... 45 more Caused by: GSSException: No valid credentials provided (Mechanism level: Cannot get a KDC reply) at sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:715) at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:248) at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179) at org.apache.hadoop.security.authentication.client.KerberosAuthenticator$1.run(KerberosAuthenticator.java:285) at org.apache.hadoop.security.authentication.client.KerberosAuthenticator$1.run(KerberosAuthenticator.java:261) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.authentication.client.KerberosAuthenticator.doSpnegoSequence(KerberosAuthenticator.java:261) ... 49 more Caused by: java.io.IOException: Cannot get a KDC reply at sun.security.krb5.KdcComm.send(KdcComm.java:250) at sun.security.krb5.KdcComm.send(KdcComm.java:191) at sun.security.krb5.KrbTgsReq.send(KrbTgsReq.java:186) at sun.security.krb5.KrbTgsReq.sendAndGetCreds(KrbTgsReq.java:201) at sun.security.krb5.internal.CredentialsUtil.serviceCreds(CredentialsUtil.java:292) at sun.security.krb5.internal.CredentialsUtil.acquireServiceCreds(CredentialsUtil.java:101) at sun.security.krb5.Credentials.acquireServiceCreds(Credentials.java:456) at sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:641) ... 56 more 2016-02-01 06:41:48,171 ERROR namenode.FSImage (FSEditLogLoader.java:loadEditRecords(196)) - Error replaying edit log at offset 0. Expected transaction ID was 1 org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream$PrematureEOFException: got premature end-of-file at txid 0; expected file to go up to 5 at org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream.nextOp(RedundantEditLogInputStream.java:194) at org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.readOp(EditLogInputStream.java:85) at org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.skipUntil(EditLogInputStream.java:151) at org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream.nextOp(RedundantEditLogInputStream.java:178) at org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.readOp(EditLogInputStream.java:85) at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadEditRecords(FSEditLogLoader.java:187) at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadFSEdits(FSEditLogLoader.java:140) at org.apache.hadoop.hdfs.server.namenode.FSImage.loadEdits(FSImage.java:835) at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:690) at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:281) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:1063) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:767) at org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:609) at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:670) at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:838) at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:817) at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1538) at org.apache.hadoop.hdfs.MiniDFSCluster.restartNameNode(MiniDFSCluster.java:1862) at org.apache.hadoop.hdfs.MiniDFSCluster.restartNameNode(MiniDFSCluster.java:1827) at org.apache.hadoop.hdfs.qjournal.TestSecureNNWithQJM.restartNameNode(TestSecureNNWithQJM.java:197) at org.apache.hadoop.hdfs.qjournal.TestSecureNNWithQJM.doNNWithQJMTest(TestSecureNNWithQJM.java:179) at org.apache.hadoop.hdfs.qjournal.TestSecureNNWithQJM.testSecureMode(TestSecureNNWithQJM.java:159) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74) 2016-02-01 06:41:48,173 WARN namenode.FSNamesystem (FSNamesystem.java:loadFromDisk(769)) - Encountered exception loading fsimage org.apache.hadoop.hdfs.server.namenode.EditLogInputException: Error replaying edit log at offset 0. Expected transaction ID was 1 at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadEditRecords(FSEditLogLoader.java:200) at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadFSEdits(FSEditLogLoader.java:140) at org.apache.hadoop.hdfs.server.namenode.FSImage.loadEdits(FSImage.java:835) at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:690) at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:281) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:1063) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:767) at org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:609) at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:670) at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:838) at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:817) at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1538) at org.apache.hadoop.hdfs.MiniDFSCluster.restartNameNode(MiniDFSCluster.java:1862) at org.apache.hadoop.hdfs.MiniDFSCluster.restartNameNode(MiniDFSCluster.java:1827) at org.apache.hadoop.hdfs.qjournal.TestSecureNNWithQJM.restartNameNode(TestSecureNNWithQJM.java:197) at org.apache.hadoop.hdfs.qjournal.TestSecureNNWithQJM.doNNWithQJMTest(TestSecureNNWithQJM.java:179) at org.apache.hadoop.hdfs.qjournal.TestSecureNNWithQJM.testSecureMode(TestSecureNNWithQJM.java:159) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74) Caused by: org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream$PrematureEOFException: got premature end-of-file at txid 0; expected file to go up to 5 at org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream.nextOp(RedundantEditLogInputStream.java:194) at org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.readOp(EditLogInputStream.java:85) at org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.skipUntil(EditLogInputStream.java:151) at org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream.nextOp(RedundantEditLogInputStream.java:178) at org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.readOp(EditLogInputStream.java:85) at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadEditRecords(FSEditLogLoader.java:187) ... 27 more 2016-02-01 06:41:48,187 INFO mortbay.log (Slf4jLog.java:info(67)) - Stopped SslSocketConnectorSecure@localhost:41570 2016-02-01 06:41:48,287 INFO hdfs.MiniDFSCluster (MiniDFSCluster.java:shutdown(1744)) - Shutting down the Mini HDFS Cluster
          Hide
          xiaochen Xiao Chen added a comment -

          I'm gonna make this to be a timeout only trivial fix. Patch 1 updates the timeout to be nicer. The test runs on my local physical machine in around 20 seconds, so I think 3 mins would be a same guard on the test.

          I haven't made much progress on the kerberos related failure. Will create a separate jira and dig in more.

          Show
          xiaochen Xiao Chen added a comment - I'm gonna make this to be a timeout only trivial fix. Patch 1 updates the timeout to be nicer. The test runs on my local physical machine in around 20 seconds, so I think 3 mins would be a same guard on the test. I haven't made much progress on the kerberos related failure. Will create a separate jira and dig in more.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          0 mvndep 0m 13s Maven dependency ordering for branch
          +1 mvninstall 7m 10s trunk passed
          +1 compile 0m 43s trunk passed with JDK v1.8.0_66
          +1 compile 0m 43s trunk passed with JDK v1.7.0_91
          +1 checkstyle 0m 22s trunk passed
          +1 mvnsite 0m 55s trunk passed
          +1 mvneclipse 0m 13s trunk passed
          +1 findbugs 2m 9s trunk passed
          +1 javadoc 1m 13s trunk passed with JDK v1.8.0_66
          +1 javadoc 2m 0s trunk passed with JDK v1.7.0_91
          0 mvndep 0m 9s Maven dependency ordering for patch
          +1 mvninstall 0m 51s the patch passed
          +1 compile 0m 46s the patch passed with JDK v1.8.0_66
          +1 javac 0m 46s the patch passed
          +1 compile 0m 47s the patch passed with JDK v1.7.0_91
          +1 javac 0m 47s the patch passed
          +1 checkstyle 0m 22s the patch passed
          +1 mvnsite 1m 0s the patch passed
          +1 mvneclipse 0m 13s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 2m 29s the patch passed
          +1 javadoc 1m 19s the patch passed with JDK v1.8.0_66
          +1 javadoc 1m 59s the patch passed with JDK v1.7.0_91
          -1 unit 63m 16s hadoop-hdfs in the patch failed with JDK v1.8.0_66.
          -1 unit 55m 27s hadoop-hdfs in the patch failed with JDK v1.7.0_91.
          +1 asflicense 0m 22s Patch does not generate ASF License warnings.
          147m 25s



          Reason Tests
          JDK v1.8.0_66 Failed junit tests hadoop.hdfs.server.datanode.TestBlockScanner
            hadoop.hdfs.TestEncryptionZones
          JDK v1.7.0_91 Failed junit tests hadoop.hdfs.TestFileAppend
            hadoop.hdfs.shortcircuit.TestShortCircuitCache



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:0ca8df7
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12785890/HDFS-9745.01.patch
          JIRA Issue HDFS-9745
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 687d9a8007d8 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / ccbba4a
          Default Java 1.7.0_91
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/14352/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/14352/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt
          unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/14352/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HDFS-Build/14352/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt
          JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/14352/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Max memory used 76MB
          Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14352/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. 0 mvndep 0m 13s Maven dependency ordering for branch +1 mvninstall 7m 10s trunk passed +1 compile 0m 43s trunk passed with JDK v1.8.0_66 +1 compile 0m 43s trunk passed with JDK v1.7.0_91 +1 checkstyle 0m 22s trunk passed +1 mvnsite 0m 55s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 2m 9s trunk passed +1 javadoc 1m 13s trunk passed with JDK v1.8.0_66 +1 javadoc 2m 0s trunk passed with JDK v1.7.0_91 0 mvndep 0m 9s Maven dependency ordering for patch +1 mvninstall 0m 51s the patch passed +1 compile 0m 46s the patch passed with JDK v1.8.0_66 +1 javac 0m 46s the patch passed +1 compile 0m 47s the patch passed with JDK v1.7.0_91 +1 javac 0m 47s the patch passed +1 checkstyle 0m 22s the patch passed +1 mvnsite 1m 0s the patch passed +1 mvneclipse 0m 13s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 2m 29s the patch passed +1 javadoc 1m 19s the patch passed with JDK v1.8.0_66 +1 javadoc 1m 59s the patch passed with JDK v1.7.0_91 -1 unit 63m 16s hadoop-hdfs in the patch failed with JDK v1.8.0_66. -1 unit 55m 27s hadoop-hdfs in the patch failed with JDK v1.7.0_91. +1 asflicense 0m 22s Patch does not generate ASF License warnings. 147m 25s Reason Tests JDK v1.8.0_66 Failed junit tests hadoop.hdfs.server.datanode.TestBlockScanner   hadoop.hdfs.TestEncryptionZones JDK v1.7.0_91 Failed junit tests hadoop.hdfs.TestFileAppend   hadoop.hdfs.shortcircuit.TestShortCircuitCache Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12785890/HDFS-9745.01.patch JIRA Issue HDFS-9745 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 687d9a8007d8 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / ccbba4a Default Java 1.7.0_91 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/14352/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/14352/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/14352/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HDFS-Build/14352/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/14352/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Max memory used 76MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14352/console This message was automatically generated.
          Hide
          jlowe Jason Lowe added a comment -

          Seeing occasional timeouts for this test as well.

          +1 lgtm. Committing this.

          Show
          jlowe Jason Lowe added a comment - Seeing occasional timeouts for this test as well. +1 lgtm. Committing this.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          -1 patch 0m 5s HDFS-9745 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help.



          Subsystem Report/Notes
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12785890/HDFS-9745.01.patch
          JIRA Issue HDFS-9745
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16514/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. -1 patch 0m 5s HDFS-9745 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. Subsystem Report/Notes JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12785890/HDFS-9745.01.patch JIRA Issue HDFS-9745 Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16514/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          jlowe Jason Lowe added a comment -

          Thanks, Xiao Chen! I committed this to trunk, branch-2, branch-2.8, and branch-2.7.

          Show
          jlowe Jason Lowe added a comment - Thanks, Xiao Chen ! I committed this to trunk, branch-2, branch-2.8, and branch-2.7.
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10329 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10329/)
          HDFS-9745. TestSecureNNWithQJM#testSecureMode sometimes fails with (jlowe: rev 126d165efd80e266a8309241f3cf059e358f5019)

          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/qjournal/TestSecureNNWithQJM.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10329 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10329/ ) HDFS-9745 . TestSecureNNWithQJM#testSecureMode sometimes fails with (jlowe: rev 126d165efd80e266a8309241f3cf059e358f5019) (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/qjournal/TestSecureNNWithQJM.java
          Hide
          xiaochen Xiao Chen added a comment -

          Woot. Thanks Jason Lowe!

          Show
          xiaochen Xiao Chen added a comment - Woot. Thanks Jason Lowe !

            People

            • Assignee:
              xiaochen Xiao Chen
              Reporter:
              xiaochen Xiao Chen
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development