Uploaded image for project: 'CarbonData'
  1. CarbonData
  2. CARBONDATA-3807

Filter queries and projection queries with bloom columns are not hitting the bloom datamap.

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Not A Bug
    • None
    • 2.0.0
    • None
    • None
    • Ant cluster - opensource

    Description

      Filter queries and projection queries with bloom columns are not hitting the bloom datamap.

       Bloom datamap is unused as per plan, even though created.

      Test queries: 

      drop table if exists uniqdata;
      CREATE TABLE uniqdata (cust_id int,cust_name String,active_emui_version string, dob timestamp, doj timestamp, bigint_column1 bigint,bigint_column2 bigint,decimal_column1 decimal(30,10), decimal_column2 decimal(36,36),double_column1 double, double_column2 double,integer_column1 int) stored as carbondata;
      load data inpath 'hdfs://hacluster/user/prasanna/2000_UniqData.csv' into table uniqdata options('fileheader'='cust_id,cust_name,active_emui_version,dob,doj,bigint_column1,bigint_column2,decimal_column1,decimal_column2,double_column1,double_column2,integer_column1','bad_records_action'='force');

      create datamap datamapuniq_b1 on table uniqdata(cust_name) as 'bloomfilter' PROPERTIES ('BLOOM_SIZE'='640000', 'BLOOM_FPP'='0.00001');

      show indexes on uniqdata;

      explain select count from uniqdata where cust_name="CUST_NAME_00000"; --not hitting;

      explain select cust_name from uniqdata; --not hitting;

       

       

      Attachments

        1. bloom-filtercolumn-plan.png
          50 kB
          Prasanna Ravichandran
        2. bloom-show index.png
          28 kB
          Prasanna Ravichandran

        Activity

          People

            Unassigned Unassigned
            Prasanna Ravichandran Prasanna Ravichandran
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: