Uploaded image for project: 'CarbonData'
  1. CarbonData
  2. CARBONDATA-4143

UT with index server

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 2.2.0
    • None
    • None

    Description

      To enable to run UT with index server using flagĀ useIndexServer.

      excluded some of the test cases to not run with index server.

      To Fix below issues:
      1. With index server enabled, select query gives incorrect result with SI when parent and child table segments are not in sync.

      queries to execute:

      0: jdbc:hive2://dggphisprb50622:22550/> create table test (c1 string,c2 int,c3 string,c5 string) STORED AS carbondata;
      ---------

      Result

      ---------
      ---------
      No rows selected (0.564 seconds)
      0: jdbc:hive2://dggphisprb50622:22550/> load data inpath 'hdfs://hacluster/chetan/dest.csv' into table test;
      -------------

      Segment ID

      -------------

      0

      -------------
      1 row selected (1.764 seconds)
      0: jdbc:hive2://dggphisprb50622:22550/> create index index_test on table test (c3) AS 'carbondata';
      ---------

      Result

      ---------
      ---------
      No rows selected (2.412 seconds)
      0: jdbc:hive2://dggphisprb50622:22550/> load data inpath 'hdfs://hacluster/chetan/dest.csv' into table test;
      -------------

      Segment ID

      -------------

      1

      -------------
      1 row selected (2.839 seconds)
      0: jdbc:hive2://dggphisprb50622:22550/> select * from test where c3='dd';
      -------+----------

      c1 c2 c3 c5

      -------+----------

      d 4 dd ddd
      d 4 dd ddd

      -------+----------
      2 rows selected (3.452 seconds)
      0: jdbc:hive2://dggphisprb50622:22550/> delete from table index_test where segment.ID in(1);
      ---------

      Result

      ---------
      ---------
      No rows selected (0.413 seconds)
      0: jdbc:hive2://dggphisprb50622:22550/> select * from test where c3='dd';
      -------+----------

      c1 c2 c3 c5

      -------+----------

      d 4 dd ddd

      -------+----------
      1 row selected (3.262 seconds)
      0: jdbc:hive2://dggphisprb50622:22550/>

      Expected: to return 2 rows.

      2. When reindex is triggered, if stale files are present in the segment directory the segment file is being written with incorrect file names. (both valid index and stale mergeindex file names). As a result, duplicate data is present in SI table but there is no error/incorrect query results.

      Attachments

        Activity

          People

            Unassigned Unassigned
            Shreelekhya SHREELEKHYA GAMPA
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 8h 50m
                8h 50m