[FLINK-26877] Introduce auto type inference for filter push down

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: table-store-0.1.0
• Fix Version/s: table-store-0.1.0
• Component/s: Table Store
• Labels: pull-request-available
• Assignee: Jane Chan
• Reporter: Jane Chan
• Created: 28/Mar/22 01:24
• Updated: 28/Mar/22 06:51
• Resolved: 28/Mar/22 06:51

Description

Currently, the following query will fail when PredicateConverter generates Literal because auto type inference is not supported

CREATE TABLE managed_table (
  f0 BIGINT,
  f1 STRING
)

SELECT * FROM managed_table WHERE f0 > 100 

Stacktrace

Caused by: java.lang.RuntimeException: Failed to read ManifestEntry list concurrently
    at org.apache.flink.table.store.file.operation.FileStoreScanImpl.plan(FileStoreScanImpl.java:212)
    at org.apache.flink.table.store.connector.source.FileStoreSource.restoreEnumerator(FileStoreSource.java:147)
    at org.apache.flink.table.store.connector.source.FileStoreSource.createEnumerator(FileStoreSource.java:117)
    at org.apache.flink.runtime.source.coordinator.SourceCoordinator.start(SourceCoordinator.java:197)
    ... 33 more
Caused by: java.util.concurrent.ExecutionException: java.lang.ClassCastException
    at java.util.concurrent.ForkJoinTask.get(ForkJoinTask.java:1006)
    at org.apache.flink.table.store.file.operation.FileStoreScanImpl.plan(FileStoreScanImpl.java:210)
    ... 36 more
Caused by: java.lang.ClassCastException
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
    at java.util.concurrent.ForkJoinTask.getThrowableException(ForkJoinTask.java:598)
    at java.util.concurrent.ForkJoinTask.get(ForkJoinTask.java:1005)
    ... 37 more
Caused by: java.lang.ClassCastException: java.lang.Long cannot be cast to java.lang.Integer
    at java.lang.Integer.compareTo(Integer.java:52)
    at org.apache.flink.table.store.file.predicate.Literal.compareValueTo(Literal.java:59)
    at org.apache.flink.table.store.file.predicate.GreaterThan.test(GreaterThan.java:51)
    at org.apache.flink.table.store.file.operation.FileStoreScanImpl.filterManifestEntry(FileStoreScanImpl.java:269)
    at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:174)
    at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1374)
    at java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:580)
    at java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:270)
    at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
    at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1374)
    at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
    at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
    at java.util.stream.ReduceOps$ReduceTask.doLeaf(ReduceOps.java:747)
    at java.util.stream.ReduceOps$ReduceTask.doLeaf(ReduceOps.java:721)
    at java.util.stream.AbstractTask.compute(AbstractTask.java:316)
    at java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:731)
    at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
    at java.util.concurrent.ForkJoinTask.doInvoke(ForkJoinTask.java:401)
    at java.util.concurrent.ForkJoinTask.invoke(ForkJoinTask.java:734)
    at java.util.stream.ReduceOps$ReduceOp.evaluateParallel(ReduceOps.java:714)
    at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:233)
    at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
    at org.apache.flink.table.store.file.operation.FileStoreScanImpl.lambda$plan$3(FileStoreScanImpl.java:209)
    at java.util.concurrent.ForkJoinTask$AdaptedCallable.exec(ForkJoinTask.java:1424)
    ... 4 more 

Issue Links

• links to: GitHub Pull Request #64