[CALCITE-2483] Druid adapter, when querying Druid segment metadata, throws when row number is larger than Integer.MAX_VALUE

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 1.17.0
• Component/s: druid-adapter
• Assignee: Unassigned
• Reporter: Hongze Zhang
• Created: 22/Aug/18 07:07
• Updated: 22/Feb/19 01:20

Description

Calcite sends "merge" segment metadata query to Druid to fetch table schema, in the response if the value of "numRows" is beyound Integer.MAX_VALUE, it will fail JSON deserialization.

 

com.google.common.util.concurrent.UncheckedExecutionException: java.lang.RuntimeException: com.fasterxml.jackson.databind.JsonMappingException: Numeric value (19629299194) out of range owf int
at [Source: (sun.net.www.protocol.http.HttpURLConnection$HttpInputStream); line: 241, column: 26]
at [Source: (sun.net.www.protocol.http.HttpURLConnection$HttpInputStream); line: 241, column: 15] (through reference chain: java.util.ArrayList[0]->org.apache.calcite.adapter.druid.DruidConnectionImpl$JsonSegmentMetadata["numRows"])
at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2207)
at com.google.common.cache.LocalCache.get(LocalCache.java:3953)
at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3957)
at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4875)
at com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:4881)
at com.google.common.cache.LocalCache$LocalLoadingCache.apply(LocalCache.java:4899)
at com.google.common.collect.Maps$AsMapView.get(Maps.java:805)
at org.apache.calcite.schema.impl.AbstractSchema.getTable(AbstractSchema.java:89)
at org.apache.calcite.jdbc.CachingCalciteSchema.getImplicitTable(CachingCalciteSchema.java:137)
at org.apache.calcite.jdbc.CalciteSchema.getTable(CalciteSchema.java:288)
at org.apache.calcite.jdbc.CalciteMetaImpl.lambda$tables$8(CalciteMetaImpl.java:383)
at org.apache.calcite.linq4j.EnumerableDefaults$11$1.current(EnumerableDefaults.java:1871)
at org.apache.calcite.linq4j.Linq4j$CompositeEnumerable$1.current(Linq4j.java:498)
at org.apache.calcite.linq4j.EnumerableDefaults$24.moveNext(EnumerableDefaults.java:2767)
at org.apache.calcite.linq4j.EnumerableDefaults$13$1.moveNext(EnumerableDefaults.java:1948)
at org.apache.calcite.linq4j.EnumerableDefaults$24.moveNext(EnumerableDefaults.java:2766)
at org.apache.calcite.linq4j.Linq4j$EnumeratorIterator.next(Linq4j.java:685)
at org.apache.calcite.avatica.util.IteratorCursor.next(IteratorCursor.java:46)
at org.apache.calcite.avatica.AvaticaResultSet.next(AvaticaResultSet.java:217)
at sqlline.IncrementalRows.hasNext(IncrementalRows.java:65)
at sqlline.TableOutputFormat.print(TableOutputFormat.java:33)
at sqlline.SqlLine.print(SqlLine.java:1648)
at sqlline.Commands.metadata(Commands.java:199)
at sqlline.Commands.tables(Commands.java:332)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at sqlline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:38)
at sqlline.SqlLine.dispatch(SqlLine.java:791)
at sqlline.SqlLine.begin(SqlLine.java:668)
at sqlline.SqlLine.start(SqlLine.java:373)
at sqlline.SqlLine.main(SqlLine.java:265)
Caused by: java.lang.RuntimeException: com.fasterxml.jackson.databind.JsonMappingException: Numeric value (19629299194) out of range of int
at [Source: (sun.net.www.protocol.http.HttpURLConnection$HttpInputStream); line: 241, column: 26]
at [Source: (sun.net.www.protocol.http.HttpURLConnection$HttpInputStream); line: 241, column: 15] (through reference chain: java.util.ArrayList[0]->org.apache.calcite.adapter.druid.DruidConnectionImpl$JsonSegmentMetadata["numRows"])
at org.apache.calcite.adapter.druid.DruidConnectionImpl.metadata(DruidConnectionImpl.java:617)
at org.apache.calcite.adapter.druid.DruidSchema.table(DruidSchema.java:85)
at org.apache.calcite.adapter.druid.DruidSchema.lambda$getTableMap$0(DruidSchema.java:74)
at com.google.common.cache.CacheLoader$FunctionToCacheLoader.load(CacheLoader.java:149)
at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3542)
at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2323)
at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2286)
at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2201)
... 32 more
Caused by: com.fasterxml.jackson.databind.JsonMappingException: Numeric value (19629299194) out of range of int
at [Source: (sun.net.www.protocol.http.HttpURLConnection$HttpInputStream); line: 241, column: 26]
at [Source: (sun.net.www.protocol.http.HttpURLConnection$HttpInputStream); line: 241, column: 15] (through reference chain: java.util.ArrayList[0]->org.apache.calcite.adapter.druid.DruidConnectionImpl$JsonSegmentMetadata["numRows"])
at com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:391)
at com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:351)
at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.wrapAndThrow(BeanDeserializerBase.java:1704)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:290)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:151)
at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:286)
at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:245)
at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:27)
at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4001)
at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3072)
at org.apache.calcite.adapter.druid.DruidConnectionImpl.metadata(DruidConnectionImpl.java:578)
... 39 more
Caused by: com.fasterxml.jackson.core.JsonParseException: Numeric value (19629299194) out of range of int
at [Source: (sun.net.www.protocol.http.HttpURLConnection$HttpInputStream); line: 241, column: 26]
at com.fasterxml.jackson.core.JsonParser._constructError(JsonParser.java:1804)
at com.fasterxml.jackson.core.base.ParserMinimalBase._reportError(ParserMinimalBase.java:663)
at com.fasterxml.jackson.core.base.ParserBase.convertNumberToInt(ParserBase.java:869)
at com.fasterxml.jackson.core.base.ParserBase._parseIntValue(ParserBase.java:801)
at com.fasterxml.jackson.core.base.ParserBase.getIntValue(ParserBase.java:645)
at com.fasterxml.jackson.databind.deser.std.NumberDeserializers$IntegerDeserializer.deserialize(NumberDeserializers.java:472)
at com.fasterxml.jackson.databind.deser.std.NumberDeserializers$IntegerDeserializer.deserialize(NumberDeserializers.java:452)
at com.fasterxml.jackson.databind.deser.impl.FieldProperty.deserializeAndSet(FieldProperty.java:136)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:288)
... 46 more

Issue Links

• links to: PR #799