[ACCUMULO-3431] TabletServer doesn't exit when CustomNonBlockingServer errors

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Incomplete
• Component/s: tserver
• Assignee: Unassigned
• Reporter: Josh Elser
• Created: 17/Dec/14 23:19
• Updated: 03/Sep/21 22:15
• Resolved: 03/Sep/21 22:15

Description

In testing out ACCUMULO-2815, I've had situations where I misconfigured the Thrift server. I do get a nice message when this happens.

2014-12-17 13:28:29,085 [rpc.CustomNonBlockingServer] ERROR: run() exiting due to uncaught error
java.lang.RuntimeException: org.apache.thrift.transport.TTransportException: Cannot read. Remote side has closed. Tried to read 4 bytes, but only got 0 bytes. (This is often indicative of an internal error on the server side. Please check your server logs.)
        at org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:219)
        at org.apache.accumulo.core.rpc.UGIAssumingTransportFactory$1.run(UGIAssumingTransportFactory.java:51)
        at org.apache.accumulo.core.rpc.UGIAssumingTransportFactory$1.run(UGIAssumingTransportFactory.java:48)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:356)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1608)
        at org.apache.accumulo.core.rpc.UGIAssumingTransportFactory.getTransport(UGIAssumingTransportFactory.java:48)
        at org.apache.accumulo.server.rpc.WrappedTFramedTransportFactory.getTransport(WrappedTFramedTransportFactory.java:46)
        at org.apache.thrift.server.AbstractNonblockingServer$FrameBuffer.<init>(AbstractNonblockingServer.java:316)
        at org.apache.accumulo.server.rpc.CustomNonBlockingServer$CustomFrameBuffer.<init>(CustomNonBlockingServer.java:141)
        at org.apache.accumulo.server.rpc.CustomNonBlockingServer$SelectAcceptThread.handleAccept(CustomNonBlockingServer.java:253)
        at org.apache.accumulo.server.rpc.CustomNonBlockingServer$SelectAcceptThread.select(CustomNonBlockingServer.java:224)
        at org.apache.accumulo.server.rpc.CustomNonBlockingServer$SelectAcceptThread.run(CustomNonBlockingServer.java:183)
Caused by: org.apache.thrift.transport.TTransportException: Cannot read. Remote side has closed. Tried to read 4 bytes, but only got 0 bytes. (This is often indicative of an internal error on the server side. Please check your server logs.)
        at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)
        at org.apache.thrift.transport.TFramedTransport.readFrame(TFramedTransport.java:129)
        at org.apache.thrift.transport.TFramedTransport.read(TFramedTransport.java:101)
        at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
        at org.apache.thrift.transport.TSaslTransport.receiveSaslMessage(TSaslTransport.java:178)
        at org.apache.thrift.transport.TSaslServerTransport.handleSaslStartMessage(TSaslServerTransport.java:125)
        at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:253)
        at org.apache.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:41)
        at org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:216)
        ... 12 more

In the Master, this exits the process (as expected), but the TabletServer stays running for some reason. Most likely, it should also exit.