Uploaded image for project: 'Calcite'
  1. Calcite
  2. CALCITE-1324

Druid metadata query throws exception if there are non-standard aggregators

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Trivial
    • Resolution: Fixed
    • Affects Version/s: 1.9.0
    • Fix Version/s: 1.9.0
    • Component/s: druid
    • Labels:
      None

      Description

      If the approximate histogram aggregator is used the druid metadata query fails as the aggregator json contains unexpected fields.

      Quick patch which just ignores the fields and the type: https://github.com/remerge/calcite/commit/6752c302eb95a294380891c6de8dca6349fe1ed6

      com.google.common.util.concurrent.UncheckedExecutionException: java.lang.RuntimeException: com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException: Unrecognized field "resolution" (class org.apache.calcite.adapter.druid.DruidConnectionImpl$JsonAggregator), not marked as ignorable (3 known properties: "type", "name", "fieldName"])
       at [Source: sun.net.www.protocol.http.HttpURLConnection$HttpInputStream@39bebb16; line: 208, column: 24] (through reference chain: java.util.ArrayList[0]->org.apache.calcite.adapter.druid.JsonSegmentMetadata["aggregators"]->java.util.LinkedHashMap["bid_price_hist_eur"]->org.apache.calcite.adapter.druid.JsonAggregator["resolution"])
      	at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2203)
      	at com.google.common.cache.LocalCache.get(LocalCache.java:3937)
      	at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3941)
      	at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4824)
      	at com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:4830)
      	at com.google.common.cache.LocalCache$LocalLoadingCache.apply(LocalCache.java:4848)
      	at com.google.common.collect.Maps$AsMapView.get(Maps.java:793)
      	at org.apache.calcite.schema.impl.AbstractSchema.getTable(AbstractSchema.java:91)
      	at org.apache.calcite.jdbc.CachingCalciteSchema.getImplicitTable(CachingCalciteSchema.java:127)
      	at org.apache.calcite.jdbc.CalciteSchema.getTable(CalciteSchema.java:255)
      	at org.apache.calcite.jdbc.CalciteMetaImpl$12.apply(CalciteMetaImpl.java:417)
      	at org.apache.calcite.jdbc.CalciteMetaImpl$12.apply(CalciteMetaImpl.java:415)
      	at org.apache.calcite.linq4j.EnumerableDefaults$15$1.current(EnumerableDefaults.java:1893)
      	at org.apache.calcite.linq4j.Linq4j$CompositeEnumerable$1.current(Linq4j.java:498)
      	at org.apache.calcite.linq4j.EnumerableDefaults$33.moveNext(EnumerableDefaults.java:2805)
      	at org.apache.calcite.linq4j.EnumerableDefaults$17$1.moveNext(EnumerableDefaults.java:1970)
      	at org.apache.calcite.linq4j.EnumerableDefaults$33.moveNext(EnumerableDefaults.java:2804)
      	at org.apache.calcite.linq4j.Linq4j$EnumeratorIterator.next(Linq4j.java:673)
      	at org.apache.calcite.avatica.util.IteratorCursor.next(IteratorCursor.java:46)
      	at org.apache.calcite.avatica.AvaticaResultSet.next(AvaticaResultSet.java:218)
      	at org.apache.calcite.jdbc.CalciteResultSet.next(CalciteResultSet.java:86)
      	at sqlline.IncrementalRows.hasNext(IncrementalRows.java:62)
      	at sqlline.TableOutputFormat.print(TableOutputFormat.java:33)
      	at sqlline.SqlLine.print(SqlLine.java:1652)
      	at sqlline.Commands.metadata(Commands.java:199)
      	at sqlline.Commands.tables(Commands.java:332)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:606)
      	at sqlline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:36)
      	at sqlline.SqlLine.dispatch(SqlLine.java:803)
      	at sqlline.SqlLine.begin(SqlLine.java:681)
      	at sqlline.SqlLine.start(SqlLine.java:398)
      	at sqlline.SqlLine.main(SqlLine.java:292)
      Caused by: java.lang.RuntimeException: com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException: Unrecognized field "resolution" (class org.apache.calcite.adapter.druid.DruidConnectionImpl$JsonAggregator), not marked as ignorable (3 known properties: "type", "name", "fieldName"])
       at [Source: sun.net.www.protocol.http.HttpURLConnection$HttpInputStream@39bebb16; line: 208, column: 24] (through reference chain: java.util.ArrayList[0]->org.apache.calcite.adapter.druid.JsonSegmentMetadata["aggregators"]->java.util.LinkedHashMap["bid_price_hist_eur"]->org.apache.calcite.adapter.druid.JsonAggregator["resolution"])
      	at com.google.common.base.Throwables.propagate(Throwables.java:160)
      	at org.apache.calcite.adapter.druid.DruidConnectionImpl.metadata(DruidConnectionImpl.java:375)
      	at org.apache.calcite.adapter.druid.DruidSchema$1.load(DruidSchema.java:72)
      	at org.apache.calcite.adapter.druid.DruidSchema$1.load(DruidSchema.java:68)
      	at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3527)
      	at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2319)
      	at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2282)
      	at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2197)
      	... 34 more
      Caused by: com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException: Unrecognized field "resolution" (class org.apache.calcite.adapter.druid.DruidConnectionImpl$JsonAggregator), not marked as ignorable (3 known properties: "type", "name", "fieldName"])
       at [Source: sun.net.www.protocol.http.HttpURLConnection$HttpInputStream@39bebb16; line: 208, column: 24] (through reference chain: java.util.ArrayList[0]->org.apache.calcite.adapter.druid.JsonSegmentMetadata["aggregators"]->java.util.LinkedHashMap["bid_price_hist_eur"]->org.apache.calcite.adapter.druid.JsonAggregator["resolution"])
      	at com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException.from(UnrecognizedPropertyException.java:51)
      	at com.fasterxml.jackson.databind.DeserializationContext.reportUnknownProperty(DeserializationContext.java:839)
      	at com.fasterxml.jackson.databind.deser.std.StdDeserializer.handleUnknownProperty(StdDeserializer.java:1045)
      	at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.handleUnknownProperty(BeanDeserializerBase.java:1352)
      	at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.handleUnknownVanilla(BeanDeserializerBase.java:1330)
      	at com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:264)
      	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:125)
      	at com.fasterxml.jackson.databind.deser.std.MapDeserializer._readAndBindStringMap(MapDeserializer.java:495)
      	at com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:341)
      	at com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:26)
      	at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:520)
      	at com.fasterxml.jackson.databind.deser.impl.FieldProperty.deserializeAndSet(FieldProperty.java:101)
      	at com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:258)
      	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:125)
      	at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:245)
      	at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:217)
      	at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:25)
      	at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:3736)
      	at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2810)
      	at org.apache.calcite.adapter.druid.DruidConnectionImpl.metadata(DruidConnectionImpl.java:360)
      	... 40 more
      

        Attachments

          Activity

            People

            • Assignee:
              julianhyde Julian Hyde
              Reporter:
              martin.karlsch Martin Karlsch
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: