[DRILL-8495] Tried to remove unmanaged buffer

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.21.1
• Fix Version/s: 1.21.2
• Assignee: Maksym Rymar
• Reporter: Maksym Rymar
• Created: 13/May/24 21:40
• Updated: 17/May/24 14:25
• Resolved: 17/May/24 14:25

Description

 

Drill throws an exception on Hive table:

  (java.lang.IllegalStateException) Tried to remove unmanaged buffer.
    org.apache.drill.exec.ops.BufferManagerImpl.replace():51
    io.netty.buffer.DrillBuf.reallocIfNeeded():101
    org.apache.drill.exec.store.hive.writers.primitive.HiveStringWriter.write():38
    org.apache.drill.exec.store.hive.readers.HiveDefaultRecordReader.readHiveRecordAndInsertIntoRecordBatch():416
    org.apache.drill.exec.store.hive.readers.HiveDefaultRecordReader.next():402
    org.apache.drill.exec.physical.impl.ScanBatch.internalNext():235
    org.apache.drill.exec.physical.impl.ScanBatch.next():299
    org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next():237
    org.apache.drill.exec.record.AbstractRecordBatch.next():109
    org.apache.drill.exec.record.AbstractRecordBatch.next():101
    org.apache.drill.exec.record.AbstractUnaryRecordBatch.innerNext():59
    org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext():93
    org.apache.drill.exec.record.AbstractRecordBatch.next():160
    org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next():237
    org.apache.drill.exec.physical.impl.BaseRootExec.next():103
    org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext():81
    org.apache.drill.exec.physical.impl.BaseRootExec.next():93
    org.apache.drill.exec.work.fragment.FragmentExecutor.lambda$run$0():321
    java.security.AccessController.doPrivileged():-2
    javax.security.auth.Subject.doAs():422
    org.apache.hadoop.security.UserGroupInformation.doAs():1899
    org.apache.drill.exec.work.fragment.FragmentExecutor.run():310
    org.apache.drill.common.SelfCleaningRunnable.run():38
    java.util.concurrent.ThreadPoolExecutor.runWorker():1149
    java.util.concurrent.ThreadPoolExecutor$Worker.run():624
    java.lang.Thread.run():748 

 

 

Reproduce:

1. Create Hive table:

create table if NOT EXISTS students(id int, name string, surname string) stored as parquet;

1. Insert a new row with 2 string values of size > 256 bytes:

insert into students values (1, 'Veeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeery long name', 'biiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiig surname');

1. Execute Drill query:

select * from hive.`students`