[DRILL-7854] When writing to S3 "WriteOperationHelper.newUploadPartRequest" throws "NoSuchMethodError"

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.18.0
• Fix Version/s: 1.19.0
• Component/s: Storage - Other

Environment

Standalone drill running in a docker environment on x86_64 (centos7, alpine 3, debian buster)

 

Drill 1.18.0 was downloaded from an official mirror

• Assignee: Unassigned
• Reporter: Joshua Pedrick
• Created: 02/Feb/21 19:27
• Updated: 10/Jun/21 06:57
• Resolved: 10/Jun/21 06:57

Description

When writing to S3(hosted on local Minio cluster) I am getting a "NoSuchMethod" error. It appears to be related to the guava version included with hadoop-3.2.1.

 

2021-02-01 21:39:17,753 [1fe78bd6-0525-196e-b3d2-25c294a604e2:frag:1:13] ERROR o.a.d.e.p.i.p.ProjectRecordBatch - ProjectRecordBatch[projector=Projector[vector2=null, selectionVectorMode=NONE], hasRemainder=false, remainderIndex=0, recordCount=0, container=org.apache.drill.exec.record.VectorContainer@3efe0be4[recordCount = 27884, schemaChanged = false, schema = BatchSchema [, ...]]2021-02-01 21:39:17,754 [1fe78bd6-0525-196e-b3d2-25c294a604e2:frag:1:13] ERROR o.a.d.e.physical.impl.BaseRootExec - Batch dump completed.2021-02-01 21:39:17,754 [1fe78bd6-0525-196e-b3d2-25c294a604e2:frag:1:13] INFO  o.a.d.e.w.fragment.FragmentExecutor - 1fe78bd6-0525-196e-b3d2-25c294a604e2:1:13: State change requested CANCELLATION_REQUESTED --> FAILED2021-02-01 21:39:25,199 [1fe78bd6-0525-196e-b3d2-25c294a604e2:frag:1:13] ERROR o.a.d.exec.server.BootStrapContext - org.apache.drill.exec.work.WorkManager$WorkerBee$1.run() leaked an exception.java.lang.NoSuchMethodError: com/google/common/base/Preconditions.checkArgument(ZLjava/lang/String;Ljava/lang/Object;J)V (loaded from <Unknown> by sun.misc.Launcher$AppClassLoader@3156fd60) called from class org.apache.hadoop.fs.s3a.WriteOperationHelper (loaded from file:/opt/drill/jars/3rdparty/hadoop-aws-3.2.1.jar by sun.misc.Launcher$AppClassLoader@3156fd60).	at org.apache.hadoop.fs.s3a.WriteOperationHelper.newUploadPartRequest(WriteOperationHelper.java:397)	at org.apache.hadoop.fs.s3a.S3ABlockOutputStream$MultiPartUpload.uploadBlockAsync(S3ABlockOutputStream.java:584)	at org.apache.hadoop.fs.s3a.S3ABlockOutputStream$MultiPartUpload.access$000(S3ABlockOutputStream.java:521)	at org.apache.hadoop.fs.s3a.S3ABlockOutputStream.uploadCurrentBlock(S3ABlockOutputStream.java:314)	at org.apache.hadoop.fs.s3a.S3ABlockOutputStream.write(S3ABlockOutputStream.java:292)	at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.write(FSDataOutputStream.java:57)	at java.io.DataOutputStream.write(DataOutputStream.java:107)	at java.io.FilterOutputStream.write(FilterOutputStream.java:97)	at org.apache.parquet.hadoop.util.HadoopPositionOutputStream.write(HadoopPositionOutputStream.java:45)	at org.apache.parquet.bytes.CapacityByteArrayOutputStream.writeToOutput(CapacityByteArrayOutputStream.java:234)	at org.apache.parquet.bytes.CapacityByteArrayOutputStream.writeTo(CapacityByteArrayOutputStream.java:247)	at org.apache.parquet.bytes.BytesInput$CapacityBAOSBytesInput.writeAllTo(BytesInput.java:421)	at org.apache.parquet.hadoop.ParquetFileWriter.writeColumnChunk(ParquetFileWriter.java:620)	at org.apache.parquet.hadoop.ParquetColumnChunkPageWriteStore$ColumnChunkPageWriter.writeToFileWriter(ParquetColumnChunkPageWriteStore.java:268)	at org.apache.parquet.hadoop.ParquetColumnChunkPageWriteStore.flushToFileWriter(ParquetColumnChunkPageWriteStore.java:89)	at org.apache.drill.exec.store.parquet.ParquetRecordWriter.flushParquetFileWriter(ParquetRecordWriter.java:737)	at org.apache.drill.exec.store.parquet.ParquetRecordWriter.flush(ParquetRecordWriter.java:435)	at org.apache.drill.exec.store.parquet.ParquetRecordWriter.cleanup(ParquetRecordWriter.java:703)	at org.apache.drill.exec.physical.impl.WriterRecordBatch.closeWriter(WriterRecordBatch.java:203)	at org.apache.drill.exec.physical.impl.WriterRecordBatch.close(WriterRecordBatch.java:221)	at org.apache.drill.common.DeferredException.suppressingClose(DeferredException.java:159)	at org.apache.drill.exec.physical.impl.BaseRootExec.close(BaseRootExec.java:169)	at org.apache.drill.exec.work.fragment.FragmentExecutor.closeOutResources(FragmentExecutor.java:408)	at org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:239)	at org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:360)	at org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)	at java.lang.Thread.run(Thread.java:823)

Issue Links

• is fixed by: DRILL-7904 Update to 30-jre Guava version
• is fixed by: HADOOP-17098 Reduce Guava dependency in Hadoop source code