Uploaded image for project: 'CarbonData'
  1. CarbonData
  2. CARBONDATA-3807

Filter queries and projection queries with bloom columns are not hitting the bloom datamap.

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Not A Bug
    • Affects Version/s: None
    • Fix Version/s: 2.0.0
    • Component/s: None
    • Labels:
      None
    • Environment:
      Ant cluster - opensource

      Description

      Filter queries and projection queries with bloom columns are not hitting the bloom datamap.

       Bloom datamap is unused as per plan, even though created.

      Test queries: 

      drop table if exists uniqdata;
      CREATE TABLE uniqdata (cust_id int,cust_name String,active_emui_version string, dob timestamp, doj timestamp, bigint_column1 bigint,bigint_column2 bigint,decimal_column1 decimal(30,10), decimal_column2 decimal(36,36),double_column1 double, double_column2 double,integer_column1 int) stored as carbondata;
      load data inpath 'hdfs://hacluster/user/prasanna/2000_UniqData.csv' into table uniqdata options('fileheader'='cust_id,cust_name,active_emui_version,dob,doj,bigint_column1,bigint_column2,decimal_column1,decimal_column2,double_column1,double_column2,integer_column1','bad_records_action'='force');

      create datamap datamapuniq_b1 on table uniqdata(cust_name) as 'bloomfilter' PROPERTIES ('BLOOM_SIZE'='640000', 'BLOOM_FPP'='0.00001');

      show indexes on uniqdata;

      explain select count from uniqdata where cust_name="CUST_NAME_00000"; --not hitting;

      explain select cust_name from uniqdata; --not hitting;

       

       

        Attachments

        1. bloom-show index.png
          28 kB
          Prasanna Ravichandran
        2. bloom-filtercolumn-plan.png
          50 kB
          Prasanna Ravichandran

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              Prasanna Ravichandran Prasanna Ravichandran
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: