[ARROW-7840] [Java] [Integration] Java executables fail

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Cannot Reproduce
• Fix Version/s: 1.0.0
• Component/s: Integration, Java
• External issue URL: https://github.com/apache/arrow/issues/17058
• Assignee: Unassigned
• Reporter: Antoine Pitrou
• Created: 12/Feb/20 15:59
• Updated: 11/Jan/23 07:56
• Resolved: 09/Apr/20 22:45

Description

When trying to run integration tests using docker-compose run conda-integration, I always get failures during the Java tests:

RuntimeError: Command failed: ['java', '-Dio.netty.tryReflectionSetAccessible=true', '-cp', '/arrow/java/tools/target/arrow-tools-1.0.0-SNAPSHOT-jar-with-dependencies.jar', 'org.apache.arrow.tools.StreamToFile', '/tmp/tmpqbkrmpo1/e75ed336_simple.producer_file_as_stream', '/tmp/tmpqbkrmpo1/e75ed336_simple.consumer_stream_as_file']
With output:
--------------
15:57:01.194 [main] DEBUG io.netty.util.internal.logging.InternalLoggerFactory - Using SLF4J as the default logging framework
15:57:01.196 [main] DEBUG io.netty.util.ResourceLeakDetector - -Dio.netty.leakDetection.level: simple
15:57:01.196 [main] DEBUG io.netty.util.ResourceLeakDetector - -Dio.netty.leakDetection.targetRecords: 4
15:57:01.208 [main] DEBUG io.netty.util.internal.PlatformDependent0 - -Dio.netty.noUnsafe: false
15:57:01.209 [main] DEBUG io.netty.util.internal.PlatformDependent0 - Java version: 8
15:57:01.210 [main] DEBUG io.netty.util.internal.PlatformDependent0 - sun.misc.Unsafe.theUnsafe: available
15:57:01.210 [main] DEBUG io.netty.util.internal.PlatformDependent0 - sun.misc.Unsafe.copyMemory: available
15:57:01.210 [main] DEBUG io.netty.util.internal.PlatformDependent0 - java.nio.Buffer.address: available
15:57:01.210 [main] DEBUG io.netty.util.internal.PlatformDependent0 - direct buffer constructor: available
15:57:01.211 [main] DEBUG io.netty.util.internal.PlatformDependent0 - java.nio.Bits.unaligned: available, true
15:57:01.211 [main] DEBUG io.netty.util.internal.PlatformDependent0 - jdk.internal.misc.Unsafe.allocateUninitializedArray(int): unavailable prior to Java9
15:57:01.211 [main] DEBUG io.netty.util.internal.PlatformDependent0 - java.nio.DirectByteBuffer.<init>(long, int): available
15:57:01.211 [main] DEBUG io.netty.util.internal.PlatformDependent - sun.misc.Unsafe: available
15:57:01.211 [main] DEBUG io.netty.util.internal.PlatformDependent - -Dio.netty.tmpdir: /tmp (java.io.tmpdir)
15:57:01.211 [main] DEBUG io.netty.util.internal.PlatformDependent - -Dio.netty.bitMode: 64 (sun.arch.data.model)
15:57:01.212 [main] DEBUG io.netty.util.internal.PlatformDependent - -Dio.netty.noPreferDirect: false
15:57:01.212 [main] DEBUG io.netty.util.internal.PlatformDependent - -Dio.netty.maxDirectMemory: 11252269056 bytes
15:57:01.212 [main] DEBUG io.netty.util.internal.PlatformDependent - -Dio.netty.uninitializedArrayAllocationThreshold: -1
15:57:01.213 [main] DEBUG io.netty.util.internal.CleanerJava6 - java.nio.ByteBuffer.cleaner(): available
15:57:01.213 [main] DEBUG io.netty.buffer.PooledByteBufAllocator - -Dio.netty.allocator.numHeapArenas: 48
15:57:01.213 [main] DEBUG io.netty.buffer.PooledByteBufAllocator - -Dio.netty.allocator.numDirectArenas: 48
15:57:01.213 [main] DEBUG io.netty.buffer.PooledByteBufAllocator - -Dio.netty.allocator.pageSize: 8192
15:57:01.213 [main] DEBUG io.netty.buffer.PooledByteBufAllocator - -Dio.netty.allocator.maxOrder: 11
15:57:01.213 [main] DEBUG io.netty.buffer.PooledByteBufAllocator - -Dio.netty.allocator.chunkSize: 16777216
15:57:01.213 [main] DEBUG io.netty.buffer.PooledByteBufAllocator - -Dio.netty.allocator.tinyCacheSize: 512
15:57:01.213 [main] DEBUG io.netty.buffer.PooledByteBufAllocator - -Dio.netty.allocator.smallCacheSize: 256
15:57:01.213 [main] DEBUG io.netty.buffer.PooledByteBufAllocator - -Dio.netty.allocator.normalCacheSize: 64
15:57:01.213 [main] DEBUG io.netty.buffer.PooledByteBufAllocator - -Dio.netty.allocator.maxCachedBufferCapacity: 32768
15:57:01.213 [main] DEBUG io.netty.buffer.PooledByteBufAllocator - -Dio.netty.allocator.cacheTrimInterval: 8192
15:57:01.213 [main] DEBUG io.netty.buffer.PooledByteBufAllocator - -Dio.netty.allocator.useCacheForAllThreads: true
15:57:01.216 [main] DEBUG io.netty.util.internal.InternalThreadLocalMap - -Dio.netty.threadLocalMap.stringBuilder.initialSize: 1024
15:57:01.216 [main] DEBUG io.netty.util.internal.InternalThreadLocalMap - -Dio.netty.threadLocalMap.stringBuilder.maxSize: 4096
15:57:01.228 [main] DEBUG io.netty.buffer.AbstractByteBuf - -Dio.netty.buffer.bytebuf.checkAccessible: true
15:57:01.228 [main] DEBUG io.netty.util.ResourceLeakDetectorFactory - Loaded default ResourceLeakDetector: io.netty.util.ResourceLeakDetector@71bc1ae4
15:57:01.242 [main] DEBUG org.apache.arrow.vector.ipc.ReadChannel - Reading buffer with size: 4
15:57:01.242 [main] DEBUG org.apache.arrow.vector.ipc.ReadChannel - Reading buffer with size: 4
15:57:01.242 [main] DEBUG org.apache.arrow.vector.ipc.ReadChannel - Reading buffer with size: 216
15:57:01.497 [main] DEBUG org.apache.arrow.vector.ipc.ReadChannel - Reading buffer with size: 4
15:57:01.497 [main] DEBUG org.apache.arrow.vector.ipc.ReadChannel - Reading buffer with size: 4
15:57:01.497 [main] DEBUG org.apache.arrow.vector.ipc.ReadChannel - Reading buffer with size: 248
Exception in thread "main" java.lang.NoSuchMethodError: org.apache.arrow.vector.ipc.message.MessageSerializer.readMessageBody(Lorg/apache/arrow/vector/ipc/ReadChannel;ILorg/apache/arrow/memory/BufferAllocator;)Lio/netty/buffer/ArrowBuf;
	at org.apache.arrow.vector.ipc.message.MessageChannelReader.readNext(MessageChannelReader.java:68)
	at org.apache.arrow.vector.ipc.ArrowStreamReader.loadNextBatch(ArrowStreamReader.java:106)
	at org.apache.arrow.tools.StreamToFile.convert(StreamToFile.java:47)
	at org.apache.arrow.tools.StreamToFile.main(StreamToFile.java:73)