Uploaded image for project: 'Ignite'
  1. Ignite
  2. IGNITE-7939

.NET client node throws IgniteClientDisconnectedCheckedException when listening to EVT_CLIENT_NODE_DISCONNECTED and all Java server nodes exit

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: 2.3
    • Fix Version/s: None
    • Component/s: messaging, platforms
    • Labels:
      None
    • Environment:

      .NET and JVM Ignite nodes running on the same Windows 7 6.1 amd64 machine.

      Server node: Ignite JVM

      Client node: Ignite.NET

      Description

      I'm using the following Java code to start an Ignite server node:

      IgniteDotNetEventExample.java
      import org.apache.ignite.Ignite;
      import org.apache.ignite.Ignition;
      import org.apache.ignite.configuration.DataRegionConfiguration;
      import org.apache.ignite.configuration.DataStorageConfiguration;
      import org.apache.ignite.configuration.IgniteConfiguration;
      import org.apache.ignite.events.EventType;
      import org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi;
      import org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder;
      
      import java.io.IOException;
      import java.util.Collections;
      
      public class IgniteDotNetEventExample {
          @SuppressWarnings("Duplicates")
          private static Ignite startLocalIgniteServer() {
              IgniteConfiguration cfg = new IgniteConfiguration();
              // Do not persist the database to the disk.
              cfg.setDataStorageConfiguration(new DataStorageConfiguration().setDefaultDataRegionConfiguration(new DataRegionConfiguration().setPersistenceEnabled(false)));
              // Only discover other cluster nodes on the same machine.
              cfg.setDiscoverySpi(new TcpDiscoverySpi().setIpFinder(new TcpDiscoveryVmIpFinder(true).setAddresses(Collections.singletonList("127.0.0.1"))));
              cfg.setIncludeEventTypes(EventType.EVT_CLIENT_NODE_DISCONNECTED, EventType.EVT_CLIENT_NODE_RECONNECTED);
              cfg.setClientMode(false);
      
              return Ignition.start(cfg);
          }
      
          @SuppressWarnings({"try","unused"})
          public static void main(String[] args) throws IOException {
              try (Ignite ignite = startLocalIgniteServer()) {
                  //noinspection ResultOfMethodCallIgnored
                  System.in.read();
              }
          }
      }
      

      I'm using the following C# code to start an Ignite client node:

      IgniteDotNetEventExample.cs
      using Apache.Ignite.Core;
      using Apache.Ignite.Core.Configuration;
      using Apache.Ignite.Core.Discovery.Tcp;
      using Apache.Ignite.Core.Discovery.Tcp.Static;
      using Apache.Ignite.Core.Events;
      using System;
      using System.Threading;
      
      internal class DelegateBackedEventListener<T> : IEventListener<T> where T : IEvent
      {
          private Func<T, bool> handler;
      
          internal DelegateBackedEventListener(Func<T, bool> handler)
          {
              this.handler = handler;
          }
      
          public bool Invoke(T evt)
          {
              return handler(evt);
          }
      }
      
      class IgniteDotNetEventExample
      {
          private static IIgnite StartLocalIgniteServer()
          {
              if (Environment.GetEnvironmentVariable("JAVA_HOME") == null)
              {
                  if (Environment.Is64BitProcess)
                      Environment.SetEnvironmentVariable("JAVA_HOME", @"C:\Program Files\jdk\1.8");
                  else
                      Environment.SetEnvironmentVariable("JAVA_HOME", @"C:\Program Files (x86)\jdk\1.8");
              }
              var cfg = new IgniteConfiguration()
              {
                  DataStorageConfiguration = new DataStorageConfiguration()
                  {
                      DefaultDataRegionConfiguration = new DataRegionConfiguration()
                      {
                          Name = "default",
                          PersistenceEnabled = false
                      }
                  },
                  DiscoverySpi = new TcpDiscoverySpi
                  {
                      IpFinder = new TcpDiscoveryStaticIpFinder
                      {
                          Endpoints = new string[] { "127.0.0.1" }
                      }
                  },
                  IncludedEventTypes = new int[] {
                      EventType.ClientNodeDisconnected, EventType.ClientNodeReconnected
                  },
                  ClientMode = true
              };
      
              return Ignition.Start(cfg);
          }
      
          public static void Main(string[] args)
          {
              using (IIgnite ignite = StartLocalIgniteServer())
              {
                  ignite.GetEvents().LocalListen(new DelegateBackedEventListener<DiscoveryEvent>(evt =>
                  {
                      Console.WriteLine(evt.Name + " " + evt.GetType());
                      return true;
                  }), EventType.ClientNodeDisconnected, EventType.ClientNodeReconnected);
      
                  Console.Read();
              }
          }
      }
      

      When I start the JVM server node and wait until it gives me the "Topology snapshot" message, then start the .NET client node and wait until it gives me the "Topology snapshot" message, and then terminate the server node, I receive the following error in the .NET client node:

      Mar 13, 2018 5:07:50 PM java.util.logging.LogManager$RootLogger log
      SEVERE: Failed to resolve default logging config file: config/java.util.logging.
      properties
      [17:07:50]    __________  ________________
      [17:07:50]   /  _/ ___/ |/ /  _/_  __/ __/
      [17:07:50]  _/ // (7 7    // /  / / / _/
      [17:07:50] /___/\___/_/|_/___/ /_/ /___/
      [17:07:50]
      [17:07:50] ver. 2.3.0#20171028-sha1:8add7fd5
      [17:07:50] 2017 Copyright(C) Apache Software Foundation
      [17:07:50]
      [17:07:50] Ignite documentation: http://ignite.apache.org
      [17:07:50]
      [17:07:50] Quiet mode.
      [17:07:50]   ^-- To see **FULL** console log here add -DIGNITE_QUIET=false or "-v" to ignite.{sh|bat}
      [17:07:50]
      [17:07:50] OS: Windows 7 6.1 amd64
      [17:07:50] VM information: Java(TM) SE Runtime Environment 1.8.0_45-b15 Oracle Corporation Java HotSpot(TM) 64-Bit Server VM 25.45-b02
      [17:07:50] Initial heap size is 384MB (should be no less than 512MB, use -Xms512m -Xmx512m).
      [17:07:50] Configured plugins:
      [17:07:50]   ^-- None
      [17:07:50]
      [17:07:51] Message queue limit is set to 0 which may lead to potential OOMEs when running cache operations in FULL_ASYNC or PRIMARY_SYNC modes due to message queues growth on sender and receiver sides.
      [17:07:51] Security status [authentication=off, tls/ssl=off]
      [17:07:52] REST protocols do not start on client node. To start the protocols on client node set '-DIGNITE_REST_START_ON_CLIENT=true' system property.
      [17:07:54] Failed to connect to any address from IP finder (will retry to join topology every 2 secs): [/127.0.0.1:47500]
      [17:08:13] Performance suggestions for grid  (fix if possible)
      [17:08:13] To disable, set -DIGNITE_PERFORMANCE_SUGGESTIONS_DISABLED=true
      [17:08:13]   ^-- Disable grid events (remove 'includeEventTypes' from configuration)
      [17:08:13]   ^-- Enable G1 Garbage Collector (add '-XX:+UseG1GC' to JVM options)
      [17:08:13]   ^-- Specify JVM heap max size (add '-Xmx<size>[g|G|m|M|k|K]' to JVM options)
      [17:08:13]   ^-- Set max direct memory size if getting 'OOME: Direct buffer memory' (add '-XX:MaxDirectMemorySize=<size>[g|G|m|M|k|K]' to JVM options)
      [17:08:13]   ^-- Disable processing of calls to System.gc() (add '-XX:+DisableExplicitGC' to JVM options)
      [17:08:13] Refer to this page for more performance suggestions: https://apacheignite.readme.io/docs/jvm-and-system-tuning
      [17:08:13]
      [17:08:13] To start Console Management & Monitoring run ignitevisorcmd.{sh|bat}
      [17:08:13]
      [17:08:13] Ignite node started OK (id=7f264dc6)
      [17:08:13] Topology snapshot [ver=2, servers=1, clients=1, CPUs=4, heap=6.4GB]
      Mar 13, 2018 5:08:40 PM org.apache.ignite.logger.java.JavaLogger error
      SEVERE: Failed to send message: TcpDiscoveryClientMetricsUpdateMessage [super=TcpDiscoveryAbstractMessage [sndNodeId=null, id=59ec0312261-7f264dc6-e03c-43ff-810e-26df357361cc, verifierNodeId=null, topVer=0, pendingIdx=0, failedNodes=null, isClient=true]]
      java.net.SocketException: Socket is closed
          at java.net.Socket.getSendBufferSize(Socket.java:1215)
          at org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.socketStream(TcpDiscoverySpi.java:1358)
          at org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.writeToSocket(TcpDiscoverySpi.java:1470)
          at org.apache.ignite.spi.discovery.tcp.ClientImpl$SocketWriter.body(ClientImpl.java:1240)
          at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)
      
      Mar 13, 2018 5:08:45 PM org.apache.ignite.logger.java.JavaLogger error
      SEVERE: Failed to reconnect to cluster (consider increasing 'networkTimeout' configuration property) [networkTimeout=5000]
      Mar 13, 2018 5:08:45 PM org.apache.ignite.logger.java.JavaLogger error
      SEVERE: Unexpected exception in listener notification for event: DiscoveryEvent [evtNode=TcpDiscoveryNode [id=19a3aa4e-7da7-4a7a-ae09-5ffb60929b65, addrs=[0:0:0:0:0:0:0:1, 10.54.158.151, 127.0.0.1], sockAddrs=[dcnds0057251.firmwide.corp.gs.com/10.54.158.151:0, /0:0:0:0:0:0:0:1:0, /127.0.0.1:0], discPort=0, order=2, intOrder=0, lastExchangeTime=1520975272485, loc=true, ver=2.3.0#20171028-sha1:8add7fd5, isClient=true], topVer=2, nodeId8=19a3aa4e, msg=Client node disconnected: TcpDiscoveryNode [id=7f264dc6-e03c-43ff-810e-26df357361cc, addrs=[0:0:0:0:0:0:0:1, 10.54.158.151, 127.0.0.1], sockAddrs=[dcnds0057251.firmwide.corp.gs.com/10.54.158.151:0, /0:0:0:0:0:0:0:1:0, /127.0.0.1:0], discPort=0, order=2, intOrder=0, lastExchangeTime=1520975272485, loc=true, ver=2.3.0#20171028-sha1:8add7fd5, isClient=true], type=CLIENT_NODE_DISCONNECTED, tstamp=1520975325184]
      class org.apache.ignite.binary.BinaryObjectException: Failed to update meta data for type: org.apache.ignite.lang.IgniteUuid
          at org.apache.ignite.internal.processors.cache.binary.CacheObjectBinaryProcessorImpl.addMeta(CacheObjectBinaryProcessorImpl.java:449)
          at org.apache.ignite.internal.processors.cache.binary.CacheObjectBinaryProcessorImpl$2.addMeta(CacheObjectBinaryProcessorImpl.java:182)
          at org.apache.ignite.internal.binary.BinaryContext.updateMetadata(BinaryContext.java:1304)
          at org.apache.ignite.internal.binary.BinaryClassDescriptor.write(BinaryClassDescriptor.java:762)
          at org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal0(BinaryWriterExImpl.java:206)
          at org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:147)
          at org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:134)
          at org.apache.ignite.internal.binary.BinaryWriterExImpl.doWriteObject(BinaryWriterExImpl.java:496)
          at org.apache.ignite.internal.binary.BinaryWriterExImpl.writeObject(BinaryWriterExImpl.java:1469)
          at org.apache.ignite.internal.processors.platform.PlatformContextImpl.writeCommonEventData(PlatformContextImpl.java:545)
          at org.apache.ignite.internal.processors.platform.PlatformContextImpl.writeEvent(PlatformContextImpl.java:500)
          at org.apache.ignite.internal.processors.platform.events.PlatformEventFilterListenerImpl.apply0(PlatformEventFilterListenerImpl.java:130)
          at org.apache.ignite.internal.processors.platform.events.PlatformEventFilterListenerImpl.apply(PlatformEventFilterListenerImpl.java:93)
          at org.apache.ignite.internal.processors.platform.events.PlatformEventFilterListenerImpl.apply(PlatformEventFilterListenerImpl.java:38)
          at org.apache.ignite.internal.managers.eventstorage.GridEventStorageManager$UserListenerWrapper.onEvent(GridEventStorageManager.java:1477)
          at org.apache.ignite.internal.managers.eventstorage.GridEventStorageManager.notifyListeners(GridEventStorageManager.java:873)
          at org.apache.ignite.internal.managers.eventstorage.GridEventStorageManager.notifyListeners(GridEventStorageManager.java:858)
          at org.apache.ignite.internal.managers.eventstorage.GridEventStorageManager.record0(GridEventStorageManager.java:341)
          at org.apache.ignite.internal.managers.eventstorage.GridEventStorageManager.record(GridEventStorageManager.java:307)
          at org.apache.ignite.internal.managers.discovery.GridDiscoveryManager$DiscoveryWorker.recordEvent(GridDiscoveryManager.java:2505)
          at org.apache.ignite.internal.managers.discovery.GridDiscoveryManager$DiscoveryWorker.body0(GridDiscoveryManager.java:2711)
          at org.apache.ignite.internal.managers.discovery.GridDiscoveryManager$DiscoveryWorker.body(GridDiscoveryManager.java:2534)
          at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
          at java.lang.Thread.run(Thread.java:745)
      Caused by: class org.apache.ignite.internal.IgniteClientDisconnectedCheckedException: Failed to send custom message: client is disconnected.
          at org.apache.ignite.internal.managers.discovery.GridDiscoveryManager.sendCustomEvent(GridDiscoveryManager.java:2101)
          at org.apache.ignite.internal.processors.cache.binary.BinaryMetadataTransport.requestMetadataUpdate(BinaryMetadataTransport.java:173)
          at org.apache.ignite.internal.processors.cache.binary.CacheObjectBinaryProcessorImpl.addMeta(CacheObjectBinaryProcessorImpl.java:441)
          ... 23 more
      

      The expected behavior is for the Console.WriteLine(evt.Name + " " + evt.GetType()) line to be executed, but that evidently did not occur.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              goldenkevin Kevin Jin
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:

                Time Tracking

                Estimated:
                Original Estimate - 672h
                672h
                Remaining:
                Remaining Estimate - 672h
                672h
                Logged:
                Time Spent - Not Specified
                Not Specified