Uploaded image for project: 'Calcite'
  1. Calcite
  2. CALCITE-2483

Druid adapter, when querying Druid segment metadata, throws when row number is larger than Integer.MAX_VALUE

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 1.17.0
    • Fix Version/s: None
    • Component/s: druid-adapter
    • Labels:
      None

      Description

      Calcite sends "merge" segment metadata query to Druid to fetch table schema, in the response if the value of "numRows" is beyound Integer.MAX_VALUE, it will fail JSON deserialization.

       

      com.google.common.util.concurrent.UncheckedExecutionException: java.lang.RuntimeException: com.fasterxml.jackson.databind.JsonMappingException: Numeric value (19629299194) out of range owf int
      at [Source: (sun.net.www.protocol.http.HttpURLConnection$HttpInputStream); line: 241, column: 26]
      at [Source: (sun.net.www.protocol.http.HttpURLConnection$HttpInputStream); line: 241, column: 15] (through reference chain: java.util.ArrayList[0]->org.apache.calcite.adapter.druid.DruidConnectionImpl$JsonSegmentMetadata["numRows"])
      at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2207)
      at com.google.common.cache.LocalCache.get(LocalCache.java:3953)
      at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3957)
      at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4875)
      at com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:4881)
      at com.google.common.cache.LocalCache$LocalLoadingCache.apply(LocalCache.java:4899)
      at com.google.common.collect.Maps$AsMapView.get(Maps.java:805)
      at org.apache.calcite.schema.impl.AbstractSchema.getTable(AbstractSchema.java:89)
      at org.apache.calcite.jdbc.CachingCalciteSchema.getImplicitTable(CachingCalciteSchema.java:137)
      at org.apache.calcite.jdbc.CalciteSchema.getTable(CalciteSchema.java:288)
      at org.apache.calcite.jdbc.CalciteMetaImpl.lambda$tables$8(CalciteMetaImpl.java:383)
      at org.apache.calcite.linq4j.EnumerableDefaults$11$1.current(EnumerableDefaults.java:1871)
      at org.apache.calcite.linq4j.Linq4j$CompositeEnumerable$1.current(Linq4j.java:498)
      at org.apache.calcite.linq4j.EnumerableDefaults$24.moveNext(EnumerableDefaults.java:2767)
      at org.apache.calcite.linq4j.EnumerableDefaults$13$1.moveNext(EnumerableDefaults.java:1948)
      at org.apache.calcite.linq4j.EnumerableDefaults$24.moveNext(EnumerableDefaults.java:2766)
      at org.apache.calcite.linq4j.Linq4j$EnumeratorIterator.next(Linq4j.java:685)
      at org.apache.calcite.avatica.util.IteratorCursor.next(IteratorCursor.java:46)
      at org.apache.calcite.avatica.AvaticaResultSet.next(AvaticaResultSet.java:217)
      at sqlline.IncrementalRows.hasNext(IncrementalRows.java:65)
      at sqlline.TableOutputFormat.print(TableOutputFormat.java:33)
      at sqlline.SqlLine.print(SqlLine.java:1648)
      at sqlline.Commands.metadata(Commands.java:199)
      at sqlline.Commands.tables(Commands.java:332)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      at java.lang.reflect.Method.invoke(Method.java:498)
      at sqlline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:38)
      at sqlline.SqlLine.dispatch(SqlLine.java:791)
      at sqlline.SqlLine.begin(SqlLine.java:668)
      at sqlline.SqlLine.start(SqlLine.java:373)
      at sqlline.SqlLine.main(SqlLine.java:265)
      Caused by: java.lang.RuntimeException: com.fasterxml.jackson.databind.JsonMappingException: Numeric value (19629299194) out of range of int
      at [Source: (sun.net.www.protocol.http.HttpURLConnection$HttpInputStream); line: 241, column: 26]
      at [Source: (sun.net.www.protocol.http.HttpURLConnection$HttpInputStream); line: 241, column: 15] (through reference chain: java.util.ArrayList[0]->org.apache.calcite.adapter.druid.DruidConnectionImpl$JsonSegmentMetadata["numRows"])
      at org.apache.calcite.adapter.druid.DruidConnectionImpl.metadata(DruidConnectionImpl.java:617)
      at org.apache.calcite.adapter.druid.DruidSchema.table(DruidSchema.java:85)
      at org.apache.calcite.adapter.druid.DruidSchema.lambda$getTableMap$0(DruidSchema.java:74)
      at com.google.common.cache.CacheLoader$FunctionToCacheLoader.load(CacheLoader.java:149)
      at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3542)
      at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2323)
      at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2286)
      at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2201)
      ... 32 more
      Caused by: com.fasterxml.jackson.databind.JsonMappingException: Numeric value (19629299194) out of range of int
      at [Source: (sun.net.www.protocol.http.HttpURLConnection$HttpInputStream); line: 241, column: 26]
      at [Source: (sun.net.www.protocol.http.HttpURLConnection$HttpInputStream); line: 241, column: 15] (through reference chain: java.util.ArrayList[0]->org.apache.calcite.adapter.druid.DruidConnectionImpl$JsonSegmentMetadata["numRows"])
      at com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:391)
      at com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:351)
      at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.wrapAndThrow(BeanDeserializerBase.java:1704)
      at com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:290)
      at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:151)
      at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:286)
      at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:245)
      at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:27)
      at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4001)
      at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3072)
      at org.apache.calcite.adapter.druid.DruidConnectionImpl.metadata(DruidConnectionImpl.java:578)
      ... 39 more
      Caused by: com.fasterxml.jackson.core.JsonParseException: Numeric value (19629299194) out of range of int
      at [Source: (sun.net.www.protocol.http.HttpURLConnection$HttpInputStream); line: 241, column: 26]
      at com.fasterxml.jackson.core.JsonParser._constructError(JsonParser.java:1804)
      at com.fasterxml.jackson.core.base.ParserMinimalBase._reportError(ParserMinimalBase.java:663)
      at com.fasterxml.jackson.core.base.ParserBase.convertNumberToInt(ParserBase.java:869)
      at com.fasterxml.jackson.core.base.ParserBase._parseIntValue(ParserBase.java:801)
      at com.fasterxml.jackson.core.base.ParserBase.getIntValue(ParserBase.java:645)
      at com.fasterxml.jackson.databind.deser.std.NumberDeserializers$IntegerDeserializer.deserialize(NumberDeserializers.java:472)
      at com.fasterxml.jackson.databind.deser.std.NumberDeserializers$IntegerDeserializer.deserialize(NumberDeserializers.java:452)
      at com.fasterxml.jackson.databind.deser.impl.FieldProperty.deserializeAndSet(FieldProperty.java:136)
      at com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:288)
      ... 46 more
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                zhztheplayer Hongze Zhang
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated: