Uploaded image for project: 'CarbonData'
  1. CarbonData
  2. CARBONDATA-4147

Carbondata 2.1.0 MV ERROR inserting data into table with MV

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.1.0
    • Fix Version/s: 2.1.1
    • Component/s: core
    • Environment:
      Apache carbondata 2.1.0
    • Flags:
      Important

      Description

      Hi Team ,

       

      We are working on a POC where we are using carbon 2.1.0.

      We have created below tables,  MV :

      create table if not exists fact_365_1_eutrancell_21 (ts timestamp, metric STRING, tags_id STRING, value DOUBLE) partitioned by (ts2 timestamp) stored as carbondata TBLPROPERTIES ('SORT_COLUMNS'='metric')

      create materialized view if not exists fact_365_1_eutrancell_21_30_minute as select tags_id ,metric ,ts2, timeseries(ts,'thirty_minute') as ts,sum(value),avg(value),min(value),max(value) from fact_365_1_eutrancell_21 group by metric, tags_id, timeseries(ts,'thirty_minute') ,ts2

       

      When i try to insert data into above Table, below error is thrown  :

      scala> carbon.sql("insert into fact_365_1_eutrancell_21 values ('2020-09-25 05:30:00','eUtranCell.HHO.X2.InterFreq.PrepAttOut','ff6cb0f7-fba0-4134-81ee-55e820574627',392.2345,'2020-09-25 05:30:00')").show()
      21/03/10 22:32:20 AUDIT audit: {"time":"March 10, 2021 10:32:20 PM IST","username":"root","opName":"INSERT INTO","opId":"33474031950342736","opStatus":"START"}
      [Stage 0:> (0 + 1) / 1]21/03/10 22:32:32 WARN CarbonOutputIteratorWrapper: try to poll a row batch one more time.
      21/03/10 22:32:32 WARN CarbonOutputIteratorWrapper: try to poll a row batch one more time.
      21/03/10 22:32:32 WARN CarbonOutputIteratorWrapper: try to poll a row batch one more time.
      21/03/10 22:32:36 WARN log: Updating partition stats fast for: fact_365_1_eutrancell_21
      21/03/10 22:32:36 WARN log: Updated size to 2699
      21/03/10 22:32:38 AUDIT audit: {"time":"March 10, 2021 10:32:38 PM IST","username":"root","opName":"INSERT OVERWRITE","opId":"33474049863830951","opStatus":"START"}
      [Stage 3:======================================================>(199 + 1) / 200]21/03/10 22:33:07 WARN CarbonOutputIteratorWrapper: try to poll a row batch one more time.
      21/03/10 22:33:07 WARN CarbonOutputIteratorWrapper: try to poll a row batch one more time.
      21/03/10 22:33:07 WARN CarbonOutputIteratorWrapper: try to poll a row batch one more time.
      21/03/10 22:33:07 ERROR CarbonFactDataHandlerColumnar: Error in producer
      java.lang.ClassCastException: java.lang.Double cannot be cast to java.lang.Long
      at org.apache.carbondata.core.datastore.page.ColumnPage.putData(ColumnPage.java:402)
      at org.apache.carbondata.processing.store.TablePage.convertToColumnarAndAddToPages(TablePage.java:239)
      at org.apache.carbondata.processing.store.TablePage.addRow(TablePage.java:201)
      at org.apache.carbondata.processing.store.CarbonFactDataHandlerColumnar.processDataRows(CarbonFactDataHandlerColumnar.java:397)
      at org.apache.carbondata.processing.store.CarbonFactDataHandlerColumnar.access$500(CarbonFactDataHandlerColumnar.java:60)
      at org.apache.carbondata.processing.store.CarbonFactDataHandlerColumnar$Producer.call(CarbonFactDataHandlerColumnar.java:637)
      at org.apache.carbondata.processing.store.CarbonFactDataHandlerColumnar$Producer.call(CarbonFactDataHandlerColumnar.java:614)
      at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      at java.lang.Thread.run(Thread.java:748)

       

       

      It seems the method is converting "decimal" data type of table to a "long" data type for MV.

      During value conversion it is throwing the error.

      Could you please check if this is a defect / bug or let me know if i have missed something ?

      Note : This was working in carbon 2.0.1

       

        Attachments

        1. carbondata_210_insert_error_stack-trace
          71 kB
          Sushant Sammanwar

          Activity

            People

            • Assignee:
              Indhumathi27 Indhumathi Muthumurugesh
              Reporter:
              sushantsam Sushant Sammanwar
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 3h 10m
                3h 10m