Uploaded image for project: 'CarbonData'
  1. CarbonData
  2. CARBONDATA-2550

[MV] Limit is ignored when data fetched from MV, Query rewrite is Wrong

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 1.5.0, 1.4.1
    • None
    • None

    Description

      0: jdbc:hive2://10.18.222.231:23040> create table mvtable1(name string,age int,salary int) stored by 'carbondata';
      ---------+

      Result

      ---------+
      ---------+
      No rows selected (0.279 seconds)
      0: jdbc:hive2://10.18.222.231:23040> insert into mvtable1 select 'n1',12,12;
      ---------+

      Result

      ---------+
      ---------+
      No rows selected (11.973 seconds)
      0: jdbc:hive2://10.18.222.231:23040> insert into mvtable1 select 'n1',12,12;
      ---------+

      Result

      ---------+
      ---------+
      No rows selected (9.92 seconds)
      0: jdbc:hive2://10.18.222.231:23040> insert into mvtable1 select 'n3',12,12;
      ---------+

      Result

      ---------+
      ---------+
      No rows selected (9.883 seconds)
      0: jdbc:hive2://10.18.222.231:23040> insert into mvtable1 select 'n4',12,12;
      ---------+

      Result

      ---------+
      ---------+
      No rows selected (10.488 seconds)
      0: jdbc:hive2://10.18.222.231:23040> select name,sum(salary) from mvtable1 group by name;
      -------------------

      name sum(salary)

      -------------------

      n3 12
      n1 24
      n4 12

      -------------------

      0: jdbc:hive2://10.18.222.231:23040> select name,sum(salary) from mvtable1 group by name limit 2;
      -------------------

      name sum(salary)

      -------------------

      n3 12
      n1 24

      -------------------
      2 rows selected (4.175 seconds)
      0: jdbc:hive2://10.18.222.231:23040> create datamap map1 using 'mv' as select name,sum(salary) from mvtable1 group by name;
      ---------+

      Result

      ---------+
      ---------+
      No rows selected (0.396 seconds)
      0: jdbc:hive2://10.18.222.231:23040> rebuild datamap map1;
      ---------+

      Result

      ---------+
      ---------+
      No rows selected (13.246 seconds)

       

      0: jdbc:hive2://10.18.222.231:23040> select name,sum(salary) from mvtable1 group by name limit 2;
      ---------------------------

      mvtable1_name sum_salary

      ---------------------------

      n3 12
      n1 24
      n4 12

      ---------------------------
      3 rows selected (2.453 seconds)
      0: jdbc:hive2://10.18.222.231:23040> select name,sum(salary) from mvtable1 group by name limit 1;
      ---------------------------

      mvtable1_name sum_salary

      ---------------------------

      n3 12
      n1 24
      n4 12

      ---------------------------
      3 rows selected (0.347 seconds)
      0: jdbc:hive2://10.18.222.231:23040>

       

       

      Even limit is given MV returns all the records from MV table.

      Cause:- 

      When Rewriting MV query ,limit is ignored,

      0: jdbc:hive2://10.18.222.231:23040> explain select name,sum(salary) from mvtable1 group by name limit 2;

      plan
      == CarbonData Profiler ==
      Table Scan on map1_table
      • total blocklets: 2
      • filter: none
      • pruned by Main DataMap
      • skipped blocklets: 0
      == Physical Plan ==
      *BatchedScan CarbonDatasourceHadoopRelation [ Database name :default, Table name :map1_table, Schema :Some(StructType(StructField(mvtable1_name,StringType,true), StructField(sum_salary,LongType,true))) ] default.map1_tablemvtable1_name#4438,sum_salary#4614L

      ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
      2 rows selected (0.36

       

       

      Attachments

        Issue Links

          Activity

            People

              xubo245 Bo Xu
              Bjangir Babulal
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 3h 50m
                  3h 50m