Uploaded image for project: 'CarbonData'
  1. CarbonData
  2. CARBONDATA-2155

IS NULL not working correctly on string datatype with dictionary_include in presto integration

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 1.3.0
    • Fix Version/s: 1.4.0
    • Component/s: presto-integration
    • Labels:
      None
    • Environment:
      Spark-2.1
      Presto 0.187

      Description

      Steps to reproduce:

      1) Create table on carbondata and load data to it.

      create table if not exists lineitem_carbon1(
      L_SHIPDATE date,
      L_SHIPMODE string,
      L_SHIPINSTRUCT string,
      L_RETURNFLAG string,
      L_RECEIPTDATE date,
      L_ORDERKEY string,
      L_PARTKEY string,
      L_SUPPKEY   string,
      L_LINENUMBER int,
      L_QUANTITY double,
      L_EXTENDEDPRICE double,
      L_DISCOUNT double,
      L_TAX double,
      L_LINESTATUS string,
      L_COMMITDATE date,
      L_COMMENT  string
      ) STORED BY 'carbondata'
      TBLPROPERTIES ('DICTIONARY_INCLUDE'='L_SHIPMODE,L_SHIPINSTRUCT,L_RETURNFLAG,L_LINESTATUS', 'table_blocksize'='300', 'no_inverted_index'='L_ORDERKEY, L_PARTKEY, L_SUPPKEY, L_COMMENT');

      load data inpath "hdfs://localhost:54310/user/hduser/input-files/lineitem.csv" into table lineitem_carbon1 options('DATEFORMAT' = 'yyyy-MM-dd','DELIMITER'='|','FILEHEADER'='L_ORDERKEY,L_PARTKEY,L_SUPPKEY,L_LINENUMBER,L_QUANTITY,L_EXTENDEDPRICE,L_DISCOUNT,L_TAX,L_RETURNFLAG,L_LINESTATUS,L_SHIPDATE,L_COMMITDATE,L_RECEIPTDATE,L_SHIPINSTRUCT,L_SHIPMODE,L_COMMENT','BAD_RECORDS_LOGGER_ENABLE'='true', 'BAD_RECORDS_ACTION'='FORCE');

      1: jdbc:hive2://localhost:10000> select l_shipmode from lineitem_carbon1 where l_shipmode is NULL;
      -------------+

      l_shipmode

      -------------+

      NULL

      -------------–+

      2) Access the same table from presto-cli and try to run select query form there:

      presto:performance> select l_shipmode from lineitem_carbon1 where l_shipmode is NULL;
      l_shipmode
      ------------
      (0 rows)

       Expected Result: It should be same as result from carbon.

        Attachments

        1. lineitem.csv
          1.22 MB
          Sangeeta Gulia

          Issue Links

            Activity

              People

              • Assignee:
                anubhavtarar anubhav tarar
                Reporter:
                sangeeta04 Sangeeta Gulia
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 4h 20m
                  4h 20m