[ARROW-14014] FlightClient.ClientStreamListener not notified on error when parsing invalid trailers

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 5.0.0
• Fix Version/s: 6.0.0
• Component/s: Java
• Labels: pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/29613
• Assignee: Bryan Cutler
• Reporter: manudebouc
• Created: 16/Sep/21 10:04
• Updated: 11/Jan/23 08:37
• Resolved: 05/Oct/21 03:38

Description

When using FlightClient.startPut combined with an AsyncPutListener, we are sometimes blocked for ever on FlightClient.ClientStreamListener.getResult() because we do not receive error notification.

Due to intermediate proxy we sometime receive 502 or 504 errors and invalid ':status' header in trailers that cannot be parsed by StatusUtils.parseTrailers in SetStreamObserver.onError(Throwable t) generating an IllegalArgumentException that prevent our listener notification, blocking for ever.

{{SEVERE: Exception while executing runnable io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed@de593f34
java.lang.IllegalArgumentException: Invalid character ':' in key name ':status'
at com.google.common.base.Preconditions.checkArgument(Preconditions.java:275)
at io.grpc.Metadata$Key.validateName(Metadata.java:742)
at io.grpc.Metadata$Key.<init>(Metadata.java:750)
at io.grpc.Metadata$Key.<init>(Metadata.java:668)
at io.grpc.Metadata$AsciiKey.<init>(Metadata.java:959)
at io.grpc.Metadata$AsciiKey.<init>(Metadata.java:954)
at io.grpc.Metadata$Key.of(Metadata.java:705)
at io.grpc.Metadata$Key.of(Metadata.java:701)
at org.apache.arrow.flight.grpc.StatusUtils.parseTrailers(StatusUtils.java:164)
at org.apache.arrow.flight.grpc.StatusUtils.fromGrpcStatusAndTrailers(StatusUtils.java:128)
at org.apache.arrow.flight.grpc.StatusUtils.fromGrpcRuntimeException(StatusUtils.java:152)
at org.apache.arrow.flight.grpc.StatusUtils.fromThrowable(StatusUtils.java:176)
at org.apache.arrow.flight.FlightClient$SetStreamObserver.onError(FlightClient.java:440)
at io.grpc.stub.ClientCalls$StreamObserverToCallListenerAdapter.onClose(ClientCalls.java:478)
at io.grpc.PartialForwardingClientCallListener.onClose(PartialForwardingClientCallListener.java:39)
at io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:23)
at io.grpc.ForwardingClientCallListener$SimpleForwardingClientCallListener.onClose(ForwardingClientCallListener.java:40)
at org.apache.arrow.flight.grpc.ClientInterceptorAdapter$FlightClientCallListener.onClose(ClientInterceptorAdapter.java:117)
at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:553)
at io.grpc.internal.ClientCallImpl.access$300(ClientCallImpl.java:68)
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInternal(ClientCallImpl.java:739)
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:718)
at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:823) }}

It seems also that same problem exsists with FlightClient.getStream() and ClientResponseObserver.onError(Throwable t)

Issue Links

• links to: GitHub Pull Request #11311