Uploaded image for project: 'CarbonData'
  1. CarbonData
  2. CARBONDATA-3472

Carbondata Integration with Presto

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 1.6.0
    • Fix Version/s: None
    • Labels:
      None
    • Environment:
      centos 7
    • Flags:
      Important

      Description

      Hi,

      I came across the below issue when I was trying to query a table stored in carbondata format through presto:

      java.lang.RuntimeException: Failed to create reader
      at org.apache.carbondata.presto.CarbondataPageSource.createReaderForColumnar(CarbondataPageSource.java:366)
      at org.apache.carbondata.presto.CarbondataPageSource.initializeForColumnar(CarbondataPageSource.java:136)
      at org.apache.carbondata.presto.CarbondataPageSource.initialize(CarbondataPageSource.java:130)
      at org.apache.carbondata.presto.CarbondataPageSource.<init>(CarbondataPageSource.java:120)
      at org.apache.carbondata.presto.CarbondataPageSourceProvider.createPageSource(CarbondataPageSourceProvider.java:88)
      at com.facebook.presto.spi.connector.classloader.ClassLoaderSafeConnectorPageSourceProvider.createPageSource(ClassLoaderSafeConnectorPageSourceProvider.java:44)
      at com.facebook.presto.split.PageSourceManager.createPageSource(PageSourceManager.java:56)
      at com.facebook.presto.operator.ScanFilterAndProjectOperator.getOutput(ScanFilterAndProjectOperator.java:221)
      at com.facebook.presto.operator.Driver.processInternal(Driver.java:379)
      at com.facebook.presto.operator.Driver.lambda$processFor$8(Driver.java:283)
      at com.facebook.presto.operator.Driver.tryWithLock(Driver.java:675)
      at com.facebook.presto.operator.Driver.processFor(Driver.java:276)
      at com.facebook.presto.execution.SqlTaskExecution$DriverSplitRunner.processFor(SqlTaskExecution.java:1077)
      at com.facebook.presto.execution.executor.PrioritizedSplitRunner.process(PrioritizedSplitRunner.java:162)
      at com.facebook.presto.execution.executor.TaskExecutor$TaskRunner.run(TaskExecutor.java:483)
      at com.facebook.presto.$gen.Presto_0_217____20190711_064626_1.run(Unknown Source)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      at java.lang.Thread.run(Thread.java:748)
      Caused by: java.io.IOException: Last dictionary chunk does not exist
      at org.apache.carbondata.core.reader.CarbonDictionaryMetadataReaderImpl.readLastEntryOfDictionaryMetaChunk(CarbonDictionaryMetadataReaderImpl.java:115)
      at org.apache.carbondata.core.cache.dictionary.AbstractDictionaryCache.readLastChunkFromDictionaryMetadataFile(AbstractDictionaryCache.java:93)
      at org.apache.carbondata.core.cache.dictionary.AbstractDictionaryCache.checkAndLoadDictionaryData(AbstractDictionaryCache.java:198)
      at org.apache.carbondata.core.cache.dictionary.ForwardDictionaryCache.getDictionary(ForwardDictionaryCache.java:212)
      at org.apache.carbondata.core.cache.dictionary.ForwardDictionaryCache.get(ForwardDictionaryCache.java:80)
      at org.apache.carbondata.core.cache.dictionary.ForwardDictionaryCache.get(ForwardDictionaryCache.java:45)
      at org.apache.carbondata.presto.CarbonDictionaryDecodeReadSupport$$anonfun$initialize$1.apply(CarbonDictionaryDecodeReadSupport.scala:65)
      at org.apache.carbondata.presto.CarbonDictionaryDecodeReadSupport$$anonfun$initialize$1.apply(CarbonDictionaryDecodeReadSupport.scala:53)
      at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
      at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:186)
      at org.apache.carbondata.presto.CarbonDictionaryDecodeReadSupport.initialize(CarbonDictionaryDecodeReadSupport.scala:53)
      at org.apache.carbondata.presto.CarbondataPageSource.createReaderForColumnar(CarbondataPageSource.java:359)
      ... 18 more

       

      This issue is seen only while querying a table which has Dictionary created on one of its columns during table creation. The same queries run fine on tables that do not have dictionaries on any of its columns.

      Please look into it.

      Thanks

       

        Attachments

        1. web_sales.txt
          10.50 MB
          Dibya

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              dibyad Dibya
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated: