Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
1.5.0
-
None
-
17 node cluster
0: jdbc:hive2://192.168.137.14:22550/default> show segments for table jinling_localsort_3;
+--------------------+----------+--------------------------+--------------------------+------------+--------------+------------+-------------+--+
| SegmentSequenceId | Status | Load Start Time | Load End Time | Merged To | File Format | Data Size | Index Size |
+--------------------+----------+--------------------------+--------------------------+------------+--------------+------------+-------------+--+
| 8 | Success | 2018-10-17 08:30:23.103 | 2018-10-17 08:30:28.476 | NA | COLUMNAR_V3 | 185.68MB | 243.36KB |
| 7 | Success | 2018-10-17 08:30:16.359 | 2018-10-17 08:30:21.667 | NA | COLUMNAR_V3 | 185.68MB | 243.36KB |
| 6 | Success | 2018-10-17 08:30:09.439 | 2018-10-17 08:30:14.97 | NA | COLUMNAR_V3 | 185.68MB | 243.36KB |
| 5 | Success | 2018-10-17 08:30:02.688 | 2018-10-17 08:30:08.036 | NA | COLUMNAR_V3 | 186.02MB | 243.35KB |
| 4 | Success | 2018-10-17 08:27:09.09 | 2018-10-17 08:27:14.553 | NA | COLUMNAR_V3 | 186.34MB | 243.35KB |
| 3 | Success | 2018-10-17 08:27:00.659 | 2018-10-17 08:27:06.096 | NA | COLUMNAR_V3 | 185.68MB | 243.36KB |
| 2 | Success | 2018-10-17 08:26:52.047 | 2018-10-17 08:26:57.47 | NA | COLUMNAR_V3 | 186.02MB | 243.35KB |
| 1 | Success | 2018-10-17 08:26:43.462 | 2018-10-17 08:26:48.899 | NA | COLUMNAR_V3 | 186.34MB | 243.35KB |
| 0 | Success | 2018-10-17 08:20:46.654 | 2018-10-17 08:20:52.681 | NA | COLUMNAR_V3 | 186.02MB | 243.35KB |
17 node cluster 0: jdbc: hive2://192.168.137.14:22550/default > show segments for table jinling_localsort_3; +--------------------+----------+--------------------------+--------------------------+------------+--------------+------------+-------------+--+ | SegmentSequenceId | Status | Load Start Time | Load End Time | Merged To | File Format | Data Size | Index Size | +--------------------+----------+--------------------------+--------------------------+------------+--------------+------------+-------------+--+ | 8 | Success | 2018-10-17 08:30:23.103 | 2018-10-17 08:30:28.476 | NA | COLUMNAR_V3 | 185.68MB | 243.36KB | | 7 | Success | 2018-10-17 08:30:16.359 | 2018-10-17 08:30:21.667 | NA | COLUMNAR_V3 | 185.68MB | 243.36KB | | 6 | Success | 2018-10-17 08:30:09.439 | 2018-10-17 08:30:14.97 | NA | COLUMNAR_V3 | 185.68MB | 243.36KB | | 5 | Success | 2018-10-17 08:30:02.688 | 2018-10-17 08:30:08.036 | NA | COLUMNAR_V3 | 186.02MB | 243.35KB | | 4 | Success | 2018-10-17 08:27:09.09 | 2018-10-17 08:27:14.553 | NA | COLUMNAR_V3 | 186.34MB | 243.35KB | | 3 | Success | 2018-10-17 08:27:00.659 | 2018-10-17 08:27:06.096 | NA | COLUMNAR_V3 | 185.68MB | 243.36KB | | 2 | Success | 2018-10-17 08:26:52.047 | 2018-10-17 08:26:57.47 | NA | COLUMNAR_V3 | 186.02MB | 243.35KB | | 1 | Success | 2018-10-17 08:26:43.462 | 2018-10-17 08:26:48.899 | NA | COLUMNAR_V3 | 186.34MB | 243.35KB | | 0 | Success | 2018-10-17 08:20:46.654 | 2018-10-17 08:20:52.681 | NA | COLUMNAR_V3 | 186.02MB | 243.35KB |
Description
If configured cache columns and min, max present flag columns is different*, it is possible to get wrong data* due to wrong index is mapped.
results inconsistent when cache is set but min/max exceeds. Column is dictionary excluded.
set carbon.minmax.allowed.byte.count=50;
0: jdbc:hive2://192.168.137.14:22550/default> use db01;
---------+
Result |
---------+
---------+
No rows selected (0.06 seconds)
0: jdbc:hive2://192.168.137.14:22550/default> show tables;
-------------------------------------------+
database | tableName | isTemporary |
-------------------------------------------+
db01 | jinling_localsort_3 | false |
-------------------------------------------+
1 row selected (0.068 seconds)
0: jdbc:hive2://192.168.137.14:22550/default> alter table jinling_localsort_3 set TBLPROPERTIES('COLUMN_META_CACHE'='user_imsi,user_num,dim24');
---------+
Result |
---------+
---------+
No rows selected (3.495 seconds)
0: jdbc:hive2://192.168.137.14:22550/default> select count from jinling_localsort_3;
-----------+
count(1) |
-----------+
9000000 |
-----------+
1 row selected (2.606 seconds)
0: jdbc:hive2://192.168.137.14:22550/default> select dim24 from jinling_localsort_3 limit 2;
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
dim24 |
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
E6Phek Hobbit from the Shire and eight companions set out on a journey to destroy the powerful One Ring and save Middle-earth from the Dark Lord Sauron.AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA |
Q4Plek Hobbit from the Shire and eight companions set out on a journey to destroy the powerful One Ring and save Middle-earth from the Dark Lord Sauron.AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA |
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
2 rows selected (457.462 seconds)
0: jdbc:hive2://192.168.137.14:22550/default> select count from jinling_localsort_3 where dim24 = 'E6Phek Hobbit from the Shire and eight companions set out on a journey to destroy the powerful One Ring and save Middle-earth from the Dark Lord Sauron.AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA';
-----------+
count(1) |
-----------+
0 |
-----------+
1 row selected (0.871 seconds)
0: jdbc:hive2://192.168.137.14:22550/default> alter table jinling_localsort_3 set TBLPROPERTIES('COLUMN_META_CACHE'='user_imsi,user_num');
---------+
Result |
---------+
---------+
No rows selected (3.798 seconds)
0: jdbc:hive2://192.168.137.14:22550/default> select count from jinling_localsort_3;
-----------+
count(1) |
-----------+
9000000 |
-----------+
1 row selected (4.162 seconds)
0: jdbc:hive2://192.168.137.14:22550/default> select count from jinling_localsort_3 where dim24 = 'E6Phek Hobbit from the Shire and eight companions set out on a journey to destroy the powerful One Ring and save Middle-earth from the Dark Lord Sauron.AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA';
-----------+
count(1) |
-----------+
90 |
-----------+
1 row selected (4.515 seconds)
Attachments
Issue Links
- links to