[FLINK-31417] Hadoop version unknown when TrinoPageSourceBase.getNextPage

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: table-store-0.4.0
• Component/s: Table Store
• Assignee: Unassigned
• Reporter: nonggia.liang
• Created: 13/Mar/23 10:07
• Updated: 19/Mar/23 05:35
• Resolved: 19/Mar/23 05:35

Description

Exception thrown when quering flink-table-store by trino

2023-03-13T11:46:36.694+0800    ERROR   SplitRunner-11-113      io.trino.execution.executor.TaskExecutor        Error processing Split 20230313_034504_00000_jdcet.1.0.0-11 {} (start = 3.599627617710298E10, wall = 89264 ms, cpu = 0 ms, wait = 1 ms, calls = 1)java.lang.NoClassDefFoundError: Could not initialize class org.apache.flink.table.store.shaded.org.apache.orc.impl.RecordReaderUtils        at org.apache.flink.table.store.shaded.org.apache.orc.impl.RecordReaderImpl.<init>(RecordReaderImpl.java:257)        at org.apache.flink.table.store.shaded.org.apache.orc.impl.ReaderImpl.rows(ReaderImpl.java:649)        at org.apache.flink.table.store.format.orc.OrcReaderFactory.createRecordReader(OrcReaderFactory.java:284)        at org.apache.flink.table.store.format.orc.OrcReaderFactory.createReader(OrcReaderFactory.java:98)        at org.apache.flink.table.store.format.orc.OrcReaderFactory.createReader(OrcReaderFactory.java:56)        at org.apache.flink.table.store.file.utils.FileUtils.createFormatReader(FileUtils.java:108)        at org.apache.flink.table.store.file.io.KeyValueDataFileRecordReader.<init>(KeyValueDataFileRecordReader.java:55)        at org.apache.flink.table.store.file.io.KeyValueFileReaderFactory.createRecordReader(KeyValueFileReaderFactory.java:95)        at org.apache.flink.table.store.file.mergetree.MergeTreeReaders.lambda$readerForRun$1(MergeTreeReaders.java:89)        at org.apache.flink.table.store.file.mergetree.compact.ConcatRecordReader.create(ConcatRecordReader.java:50)        at org.apache.flink.table.store.file.mergetree.MergeTreeReaders.readerForRun(MergeTreeReaders.java:92)        at org.apache.flink.table.store.file.mergetree.MergeTreeReaders.readerForSection(MergeTreeReaders.java:74)        at org.apache.flink.table.store.file.operation.KeyValueFileStoreRead.lambda$createReader$2(KeyValueFileStoreRead.java:195)        at org.apache.flink.table.store.file.mergetree.compact.ConcatRecordReader.create(ConcatRecordReader.java:50)        at org.apache.flink.table.store.file.operation.KeyValueFileStoreRead.createReader(KeyValueFileStoreRead.java:204)        at org.apache.flink.table.store.table.source.KeyValueTableRead.createReader(KeyValueTableRead.java:44)        at org.apache.flink.table.store.trino.TrinoPageSourceProvider.createPageSource(TrinoPageSourceProvider.java:76)        at org.apache.flink.table.store.trino.TrinoPageSourceProvider.lambda$createPageSource$0(TrinoPageSourceProvider.java:52)        at org.apache.flink.table.store.trino.ClassLoaderUtils.runWithContextClassLoader(ClassLoaderUtils.java:30)        at org.apache.flink.table.store.trino.TrinoPageSourceProvider.createPageSource(TrinoPageSourceProvider.java:51)        at io.trino.split.PageSourceManager.createPageSource(PageSourceManager.java:68)        at io.trino.operator.TableScanOperator.getOutput(TableScanOperator.java:308)        at io.trino.operator.Driver.processInternal(Driver.java:388)        at io.trino.operator.Driver.lambda$processFor$9(Driver.java:292)        at io.trino.operator.Driver.tryWithLock(Driver.java:685)        at io.trino.operator.Driver.processFor(Driver.java:285)        at io.trino.execution.SqlTaskExecution$DriverSplitRunner.processFor(SqlTaskExecution.java:1076)        at io.trino.execution.executor.PrioritizedSplitRunner.process(PrioritizedSplitRunner.java:163)        at io.trino.execution.executor.TaskExecutor$TaskRunner.run(TaskExecutor.java:488)        at io.trino.$gen.Trino_366_0____20230313_034413_2.run(Unknown Source)        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)        at java.base/java.lang.Thread.run(Thread.java:834)  2023-03-13T11:46:36.775+0800    ERROR   remote-task-callback-2  io.trino.execution.scheduler.PipelinedStageExecution    Pipelined stage execution for stage 20230313_034504_00000_jdcet.1 failedjava.lang.ExceptionInInitializerError        at org.apache.flink.table.store.shaded.org.apache.orc.impl.RecordReaderImpl.<init>(RecordReaderImpl.java:257)        at org.apache.flink.table.store.shaded.org.apache.orc.impl.ReaderImpl.rows(ReaderImpl.java:649)        at org.apache.flink.table.store.format.orc.OrcReaderFactory.createRecordReader(OrcReaderFactory.java:284)        at org.apache.flink.table.store.format.orc.OrcReaderFactory.createReader(OrcReaderFactory.java:98)        at org.apache.flink.table.store.format.orc.OrcReaderFactory.createReader(OrcReaderFactory.java:56)        at org.apache.flink.table.store.file.utils.FileUtils.createFormatReader(FileUtils.java:108)        at org.apache.flink.table.store.file.io.KeyValueDataFileRecordReader.<init>(KeyValueDataFileRecordReader.java:55)        at org.apache.flink.table.store.file.io.KeyValueFileReaderFactory.createRecordReader(KeyValueFileReaderFactory.java:95)        at org.apache.flink.table.store.file.mergetree.MergeTreeReaders.lambda$readerForRun$1(MergeTreeReaders.java:89)        at org.apache.flink.table.store.file.mergetree.compact.ConcatRecordReader.create(ConcatRecordReader.java:50)        at org.apache.flink.table.store.file.mergetree.MergeTreeReaders.readerForRun(MergeTreeReaders.java:92)        at org.apache.flink.table.store.file.mergetree.MergeTreeReaders.readerForSection(MergeTreeReaders.java:74)        at org.apache.flink.table.store.file.operation.KeyValueFileStoreRead.lambda$createReader$2(KeyValueFileStoreRead.java:195)        at org.apache.flink.table.store.file.mergetree.compact.ConcatRecordReader.readBatch(ConcatRecordReader.java:65)        at org.apache.flink.table.store.file.mergetree.DropDeleteReader.readBatch(DropDeleteReader.java:44)        at org.apache.flink.table.store.table.source.KeyValueTableRead$RowDataRecordReader.readBatch(KeyValueTableRead.java:61)        at org.apache.flink.table.store.trino.TrinoPageSourceBase.nextPage(TrinoPageSourceBase.java:120)        at org.apache.flink.table.store.trino.TrinoPageSourceBase.getNextPage(TrinoPageSourceBase.java:113)        at io.trino.operator.TableScanOperator.getOutput(TableScanOperator.java:311)        at io.trino.operator.Driver.processInternal(Driver.java:388)        at io.trino.operator.Driver.lambda$processFor$9(Driver.java:292)        at io.trino.operator.Driver.tryWithLock(Driver.java:685)        at io.trino.operator.Driver.processFor(Driver.java:285)        at io.trino.execution.SqlTaskExecution$DriverSplitRunner.processFor(SqlTaskExecution.java:1076)        at io.trino.execution.executor.PrioritizedSplitRunner.process(PrioritizedSplitRunner.java:163)        at io.trino.execution.executor.TaskExecutor$TaskRunner.run(TaskExecutor.java:488)        at io.trino.$gen.Trino_366_0____20230313_034413_2.run(Unknown Source)        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)        at java.base/java.lang.Thread.run(Thread.java:834)Caused by: java.lang.NumberFormatException: For input string: "Unknown"        at java.base/java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)        at java.base/java.lang.Integer.parseInt(Integer.java:652)        at java.base/java.lang.Integer.parseInt(Integer.java:770)        at org.apache.flink.table.store.shaded.org.apache.orc.impl.HadoopShimsFactory.get(HadoopShimsFactory.java:53)        at org.apache.flink.table.store.shaded.org.apache.orc.impl.RecordReaderUtils.<clinit>(RecordReaderUtils.java:47)        ... 30 more  2023-03-13T11:46:36.777+0800    ERROR   stage-scheduler io.trino.execution.scheduler.SqlQueryScheduler  Failure in distributed stage for query 20230313_034504_00000_jdcetjava.lang.ExceptionInInitializerError        at org.apache.flink.table.store.shaded.org.apache.orc.impl.RecordReaderImpl.<init>(RecordReaderImpl.java:257)        at org.apache.flink.table.store.shaded.org.apache.orc.impl.ReaderImpl.rows(ReaderImpl.java:649)        at org.apache.flink.table.store.format.orc.OrcReaderFactory.createRecordReader(OrcReaderFactory.java:284)        at org.apache.flink.table.store.format.orc.OrcReaderFactory.createReader(OrcReaderFactory.java:98)        at org.apache.flink.table.store.format.orc.OrcReaderFactory.createReader(OrcReaderFactory.java:56)        at org.apache.flink.table.store.file.utils.FileUtils.createFormatReader(FileUtils.java:108)        at org.apache.flink.table.store.file.io.KeyValueDataFileRecordReader.<init>(KeyValueDataFileRecordReader.java:55)        at org.apache.flink.table.store.file.io.KeyValueFileReaderFactory.createRecordReader(KeyValueFileReaderFactory.java:95)        at org.apache.flink.table.store.file.mergetree.MergeTreeReaders.lambda$readerForRun$1(MergeTreeReaders.java:89)        at org.apache.flink.table.store.file.mergetree.compact.ConcatRecordReader.create(ConcatRecordReader.java:50)        at org.apache.flink.table.store.file.mergetree.MergeTreeReaders.readerForRun(MergeTreeReaders.java:92)        at org.apache.flink.table.store.file.mergetree.MergeTreeReaders.readerForSection(MergeTreeReaders.java:74)        at org.apache.flink.table.store.file.operation.KeyValueFileStoreRead.lambda$createReader$2(KeyValueFileStoreRead.java:195)        at org.apache.flink.table.store.file.mergetree.compact.ConcatRecordReader.readBatch(ConcatRecordReader.java:65)        at org.apache.flink.table.store.file.mergetree.DropDeleteReader.readBatch(DropDeleteReader.java:44)        at org.apache.flink.table.store.table.source.KeyValueTableRead$RowDataRecordReader.readBatch(KeyValueTableRead.java:61)        at org.apache.flink.table.store.trino.TrinoPageSourceBase.nextPage(TrinoPageSourceBase.java:120)        at org.apache.flink.table.store.trino.TrinoPageSourceBase.getNextPage(TrinoPageSourceBase.java:113)        at io.trino.operator.TableScanOperator.getOutput(TableScanOperator.java:311)        at io.trino.operator.Driver.processInternal(Driver.java:388)        at io.trino.operator.Driver.lambda$processFor$9(Driver.java:292)        at io.trino.operator.Driver.tryWithLock(Driver.java:685)        at io.trino.operator.Driver.processFor(Driver.java:285)        at io.trino.execution.SqlTaskExecution$DriverSplitRunner.processFor(SqlTaskExecution.java:1076)        at io.trino.execution.executor.PrioritizedSplitRunner.process(PrioritizedSplitRunner.java:163)        at io.trino.execution.executor.TaskExecutor$TaskRunner.run(TaskExecutor.java:488)        at io.trino.$gen.Trino_366_0____20230313_034413_2.run(Unknown Source)        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)        at java.base/java.lang.Thread.run(Thread.java:834)Caused by: java.lang.NumberFormatException: For input string: "Unknown"        at java.base/java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)        at java.base/java.lang.Integer.parseInt(Integer.java:652)        at java.base/java.lang.Integer.parseInt(Integer.java:770)        at org.apache.flink.table.store.shaded.org.apache.orc.impl.HadoopShimsFactory.get(HadoopShimsFactory.java:53)        at org.apache.flink.table.store.shaded.org.apache.orc.impl.RecordReaderUtils.<clinit>(RecordReaderUtils.java:47)        ... 30 more  2023-03-13T11:46:36.784+0800    ERROR   stage-scheduler io.trino.execution.StageStateMachine    Stage 20230313_034504_00000_jdcet.1 failedjava.lang.ExceptionInInitializerError        at org.apache.flink.table.store.shaded.org.apache.orc.impl.RecordReaderImpl.<init>(RecordReaderImpl.java:257)        at org.apache.flink.table.store.shaded.org.apache.orc.impl.ReaderImpl.rows(ReaderImpl.java:649)        at org.apache.flink.table.store.format.orc.OrcReaderFactory.createRecordReader(OrcReaderFactory.java:284)        at org.apache.flink.table.store.format.orc.OrcReaderFactory.createReader(OrcReaderFactory.java:98)        at org.apache.flink.table.store.format.orc.OrcReaderFactory.createReader(OrcReaderFactory.java:56)        at org.apache.flink.table.store.file.utils.FileUtils.createFormatReader(FileUtils.java:108)        at org.apache.flink.table.store.file.io.KeyValueDataFileRecordReader.<init>(KeyValueDataFileRecordReader.java:55)        at org.apache.flink.table.store.file.io.KeyValueFileReaderFactory.createRecordReader(KeyValueFileReaderFactory.java:95)        at org.apache.flink.table.store.file.mergetree.MergeTreeReaders.lambda$readerForRun$1(MergeTreeReaders.java:89)        at org.apache.flink.table.store.file.mergetree.compact.ConcatRecordReader.create(ConcatRecordReader.java:50)        at org.apache.flink.table.store.file.mergetree.MergeTreeReaders.readerForRun(MergeTreeReaders.java:92)        at org.apache.flink.table.store.file.mergetree.MergeTreeReaders.readerForSection(MergeTreeReaders.java:74)        at org.apache.flink.table.store.file.operation.KeyValueFileStoreRead.lambda$createReader$2(KeyValueFileStoreRead.java:195)        at org.apache.flink.table.store.file.mergetree.compact.ConcatRecordReader.readBatch(ConcatRecordReader.java:65)        at org.apache.flink.table.store.file.mergetree.DropDeleteReader.readBatch(DropDeleteReader.java:44)        at org.apache.flink.table.store.table.source.KeyValueTableRead$RowDataRecordReader.readBatch(KeyValueTableRead.java:61)        at org.apache.flink.table.store.trino.TrinoPageSourceBase.nextPage(TrinoPageSourceBase.java:120)        at org.apache.flink.table.store.trino.TrinoPageSourceBase.getNextPage(TrinoPageSourceBase.java:113)        at io.trino.operator.TableScanOperator.getOutput(TableScanOperator.java:311)        at io.trino.operator.Driver.processInternal(Driver.java:388)        at io.trino.operator.Driver.lambda$processFor$9(Driver.java:292)        at io.trino.operator.Driver.tryWithLock(Driver.java:685)        at io.trino.operator.Driver.processFor(Driver.java:285)        at io.trino.execution.SqlTaskExecution$DriverSplitRunner.processFor(SqlTaskExecution.java:1076)        at io.trino.execution.executor.PrioritizedSplitRunner.process(PrioritizedSplitRunner.java:163)        at io.trino.execution.executor.TaskExecutor$TaskRunner.run(TaskExecutor.java:488)        at io.trino.$gen.Trino_366_0____20230313_034413_2.run(Unknown Source)        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)        at java.base/java.lang.Thread.run(Thread.java:834)Caused by: java.lang.NumberFormatException: For input string: "Unknown"        at java.base/java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)        at java.base/java.lang.Integer.parseInt(Integer.java:652)        at java.base/java.lang.Integer.parseInt(Integer.java:770)        at org.apache.flink.table.store.shaded.org.apache.orc.impl.HadoopShimsFactory.get(HadoopShimsFactory.java:53)        at org.apache.flink.table.store.shaded.org.apache.orc.impl.RecordReaderUtils.<clinit>(RecordReaderUtils.java:47)        ... 30 more 

Seems the common-version-info.properties file in flink-shaded-hadoop-2-uber-2.8.3-10.0.jar is not found by the classloader. The stacks tell that the call is from TrinoPageSourceBase.getNextPage, where the classloader of the current thread is AppClassloader, rather than PluginClassloader.

Can we fix it by using runWithContextClassLoader to run TrinoPageSourceBase.getNextPage with TrinoPageSourceBase.class.getClassloader?