Description
Update: Since I uncovered more problems here, I changed the title.
This fixes the following three issues:
1. PHOENIX-5550, local index is not correctly built when data existed in the data table.
2. Index ids are correctly generated on the client, so that index updates for different indexes are written into the same index.
3. Stale boundary cache is not correctly detected in some cases (namely when the scan already was in the second daughter region and that daugthter region is the last region.)
All of these happen when short view index ids are used (which is the default). The included test covers all three scenarios.
Was:
phoenix: CREATE TABLE test (pk INTEGER PRIMARY KEY, v1 INTEGER, v2 integer);
No rows affected (1.389 seconds)
phoenix: create local index l1 on test(v1);
1 row affected (11.343 seconds)
phoenix: create local index l2 on test(v1);
1 row affected (5.107 seconds)
phoenix: UPSERT INTO test VALUES(2,2,2);
1 row affected (0.037 seconds)
hbase: scan 'TEST'
ROW COLUMN+CELL
\x00\x00\xC1\x03\x00\x80\x00\x00\x02 column=L#0:\x00\x00\x00\x00, timestamp=1572842063925, value=x
\x80\x00\x00\x02 column=0:\x00\x00\x00\x00, timestamp=1572842063925, value=x
\x80\x00\x00\x02 column=0:\x80\x0B, timestamp=1572842063925, value=\x80\x00\x00\x02
\x80\x00\x00\x02 column=0:\x80\x0C, timestamp=1572842063925, value=\x80\x00\x00\x02
2 row(s) in 0.0190 seconds
There should be two index rows.
And in fact there are with phoenix.index.longViewIndex.enabled set to true, there are correctly two different index entries for both of the local indexes.
Attachments
Attachments
- 5559-test.txt
- 4 kB
- Lars Hofhansl
- 5559-4.x-HBase-1.5.txt
- 26 kB
- Lars Hofhansl
- 5559-4.x-HBase-1.5.txt
- 26 kB
- Lars Hofhansl
- 5559-4.x-HBase-1.5-v2.txt
- 27 kB
- Lars Hofhansl
- 5559-4.x-HBase-1.5-v3.txt
- 28 kB
- Lars Hofhansl
- 5559-4.x-HBase-1.5-v5.txt
- 31 kB
- Lars Hofhansl
Issue Links
- duplicates
-
PHOENIX-5550 Scan after local index creation on table having data giving wrong results when long view index id disabled
- Resolved
- links to
Activity
So I figured out what the problem is. When the client encodes the view index id it reads a value generated by the sequence. These will start with Long.MIN_VALUE.
The code eventually calls Long.shortValue(), which simply cuts off the higher bits. So instead of Long.MIN_VALUE we get 0. The next index is 1, a.s.o.
See chrajeshbabu32@gmail.com's suggestion on PHOENIX-3547 about creating two sequences (one for short ids and one for long ids), that would eliminate this problem. Alternatively we can just start the sequence ids at Short.MIN_VALUE always. We'll lose a bit of id space, but there'll still be plenty! My v3 patch on PHOENIX-5486 does that.
larsh I am fine with starting with Short.MIN_VALUE too because of still a lot of Ids space.
-v1 revives some of the code from PHOENIX-5486. The new tests pass that way.
Have a look. I'll also do more testing.
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12985146/5559-4.x-HBase-1.5.txt
against 4.x-HBase-1.5 branch at commit ec78cd142a21a2ccf6f1f53ee16b501bf81352c5.
ATTACHMENT ID: 12985146
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 release audit. The applied patch generated 1 release audit warnings (more than the master's current 0 warnings).
-1 lineLengths. The patch introduces the following lines longer than 100:
+ "CLIENT PARALLEL 1-WAY RANGE SCAN OVER IDX" + tableName + " [" + Long.toString(Short.MIN_VALUE + expectedIndexIdOffset) + ",'" + tenantId + "','" + valuePrefix + "v2-1']\n"
+ "CLIENT PARALLEL 3-WAY RANGE SCAN OVER IDX" + tableName + " [0," + Long.toString(Short.MIN_VALUE + expectedIndexIdOffset) + ",'" + tenantId + "','" + valuePrefix +
+ "v2-1'] - ["+(saltBuckets.intValue()-1)+"," + Long.toString(Short.MIN_VALUE + expectedIndexIdOffset) + ",'" + tenantId + "','" + valuePrefix + "v2-1']\n"
+ ? "CLIENT PARALLEL 1-WAY RANGE SCAN OVER " + viewIndexPhysicalName +" [" + Short.MIN_VALUE + ",51]"
+ : "CLIENT PARALLEL " + saltBuckets + "-WAY RANGE SCAN OVER " + viewIndexPhysicalName + " [0," + Short.MIN_VALUE + ",51] - ["+(saltBuckets.intValue()-1)+"," + Short.MIN_VALUE + ",51]\nCLIENT MERGE SORT",
+ ? "CLIENT PARALLEL 1-WAY RANGE SCAN OVER " + viewIndexPhysicalName +" [" + (Short.MIN_VALUE+1) + ",'foo']\n"
+ : "CLIENT PARALLEL " + saltBuckets + "-WAY RANGE SCAN OVER " + viewIndexPhysicalName + " [0," + (Short.MIN_VALUE+1) + ",'foo'] - ["+(saltBuckets.intValue()-1)+"," + (Long.MIN_VALUE+1) + ",'foo']\n"
+ + " [" + Long.toString(Short.MIN_VALUE + indexIdOffset) + ",'" + tenantId + "','f']\n" + " SERVER FILTER BY FIRST KEY ONLY",
+ ? "CLIENT PARALLEL 1-WAY RANGE SCAN OVER " + viewIndexPhysicalName + " [" + Short.MIN_VALUE + ",51]"
+ + Short.MIN_VALUE + ",51] - [" + (saltBuckets.intValue() - 1) + "," + Short.MIN_VALUE
-1 core tests. The patch failed these unit tests:
Test results: https://builds.apache.org/job/PreCommit-PHOENIX-Build/3095//testReport/
Release audit warnings: https://builds.apache.org/job/PreCommit-PHOENIX-Build/3095//artifact/patchprocess/patchReleaseAuditWarnings.txt
Console output: https://builds.apache.org/job/PreCommit-PHOENIX-Build/3095//console
This message is automatically generated.
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12985150/5559-4.x-HBase-1.5.txt
against 4.x-HBase-1.5 branch at commit ff00f67ffb2cf97adbd17cb67ff675f77a13930b.
ATTACHMENT ID: 12985150
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 release audit. The applied patch generated 1 release audit warnings (more than the master's current 0 warnings).
-1 lineLengths. The patch introduces the following lines longer than 100:
+ "CLIENT PARALLEL 1-WAY RANGE SCAN OVER IDX" + tableName + " [" + Long.toString(Short.MIN_VALUE + expectedIndexIdOffset) + ",'" + tenantId + "','" + valuePrefix + "v2-1']\n"
+ "CLIENT PARALLEL 3-WAY RANGE SCAN OVER IDX" + tableName + " [0," + Long.toString(Short.MIN_VALUE + expectedIndexIdOffset) + ",'" + tenantId + "','" + valuePrefix +
+ "v2-1'] - ["+(saltBuckets.intValue()-1)+"," + Long.toString(Short.MIN_VALUE + expectedIndexIdOffset) + ",'" + tenantId + "','" + valuePrefix + "v2-1']\n"
+ ? "CLIENT PARALLEL 1-WAY RANGE SCAN OVER " + viewIndexPhysicalName +" [" + Short.MIN_VALUE + ",51]"
+ : "CLIENT PARALLEL " + saltBuckets + "-WAY RANGE SCAN OVER " + viewIndexPhysicalName + " [0," + Short.MIN_VALUE + ",51] - ["+(saltBuckets.intValue()-1)+"," + Short.MIN_VALUE + ",51]\nCLIENT MERGE SORT",
+ ? "CLIENT PARALLEL 1-WAY RANGE SCAN OVER " + viewIndexPhysicalName +" [" + (Short.MIN_VALUE+1) + ",'foo']\n"
+ : "CLIENT PARALLEL " + saltBuckets + "-WAY RANGE SCAN OVER " + viewIndexPhysicalName + " [0," + (Short.MIN_VALUE+1) + ",'foo'] - ["+(saltBuckets.intValue()-1)+"," + (Long.MIN_VALUE+1) + ",'foo']\n"
+ + " [" + Long.toString(Short.MIN_VALUE + indexIdOffset) + ",'" + tenantId + "','f']\n" + " SERVER FILTER BY FIRST KEY ONLY",
+ ? "CLIENT PARALLEL 1-WAY RANGE SCAN OVER " + viewIndexPhysicalName + " [" + Short.MIN_VALUE + ",51]"
+ + Short.MIN_VALUE + ",51] - [" + (saltBuckets.intValue() - 1) + "," + Short.MIN_VALUE
-1 core tests. The patch failed these unit tests:
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.index.MutableIndexSplitForwardScanIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.index.MutableIndexSplitReverseScanIT
Test results: https://builds.apache.org/job/PreCommit-PHOENIX-Build/3100//testReport/
Release audit warnings: https://builds.apache.org/job/PreCommit-PHOENIX-Build/3100//artifact/patchprocess/patchReleaseAuditWarnings.txt
Console output: https://builds.apache.org/job/PreCommit-PHOENIX-Build/3100//console
This message is automatically generated.
Look 'ma, the test suite ran.
The two test failures are probably related, but it is looking very, very good.
The tests are related, but it's immediately clear how.
Somehow before splitting a local index during a scan caused a StaleRegionBoundaryCacheException, after the change this is not happening. I'm baffled.
Wow... Just changing the starting value of the sequence to a Short.MIN_VALUE causes this problem in these tests. That means that there are more problems lurking that we do not understand, yet.
Perhaps someone else can have a look too, I'm having a lot of other things on my plate.
It fails in to detect the stale boundaries BaseScannerRegionObserver.throwIfScanOutOfRegion.
When we start index ids with Long.MIN_VALUE the scan start row is:
[0, 0, 0, 0, 0, 0, 0, 0, 102, 0, 117, 0, -128, 0, 0, 20, -128, 0, 0, 21, 0]
With Short.MIN_VALUE it's:
[127, -1, -1, -1, -1, -1, -128, 0, 102, 0, 117, 0, -128, 0, 0, 20, -128, 0, 0, 21, 0]
The region start keys are 'a'-'z', and so [127, ...] happens to fall into the very last region of the table. Looks like perhaps the stale region check does not work for the last region (both the scan and that region have the [] as stop row/key)? I think that's another bug with local indexes.
rajeshbabu, mind having a look. Just apply this patch and run MutableIndexSplitForwardScanIT and you'll see the (new) problem.
So the scenario is something like:
- Start a scan using a local index.
- Split.
- Continue the scan.
- If the start of the scan happens to fall into the new last region, we won't detect the stale region boundary condition, since we only check against the region's endkey (which would not change)
Ideas?
vincentpoon, in case you have ideas.
Thanks! Putting a breakpoint in BaseScannerRegionObserver.throwIfScanOutOfRegion once with and once without the patch will show the problem.
The only difference is that different start row for the scan, where with this patch that start falls after any non-index key of for that region.
In order to find out that region boundaries change we only look at the region's end-key. For the last region of a table that might not be good enough. Can we somehow check the start key?
Here's an attempt. Uses the scan's actual start attribute set by Phoenix to check against the lower region key.
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12985400/5559-4.x-HBase-1.5-v2.txt
against 4.x-HBase-1.5 branch at commit 70baf4c6805eb46420dd1fbaf71a093ab7d84a8b.
ATTACHMENT ID: 12985400
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 release audit. The applied patch generated 1 release audit warnings (more than the master's current 0 warnings).
-1 lineLengths. The patch introduces the following lines longer than 100:
+ "CLIENT PARALLEL 1-WAY RANGE SCAN OVER IDX" + tableName + " [" + Long.toString(Short.MIN_VALUE + expectedIndexIdOffset) + ",'" + tenantId + "','" + valuePrefix + "v2-1']\n"
+ "CLIENT PARALLEL 3-WAY RANGE SCAN OVER IDX" + tableName + " [0," + Long.toString(Short.MIN_VALUE + expectedIndexIdOffset) + ",'" + tenantId + "','" + valuePrefix +
+ "v2-1'] - ["+(saltBuckets.intValue()-1)+"," + Long.toString(Short.MIN_VALUE + expectedIndexIdOffset) + ",'" + tenantId + "','" + valuePrefix + "v2-1']\n"
+ ? "CLIENT PARALLEL 1-WAY RANGE SCAN OVER " + viewIndexPhysicalName +" [" + Short.MIN_VALUE + ",51]"
+ : "CLIENT PARALLEL " + saltBuckets + "-WAY RANGE SCAN OVER " + viewIndexPhysicalName + " [0," + Short.MIN_VALUE + ",51] - ["+(saltBuckets.intValue()-1)+"," + Short.MIN_VALUE + ",51]\nCLIENT MERGE SORT",
+ ? "CLIENT PARALLEL 1-WAY RANGE SCAN OVER " + viewIndexPhysicalName +" [" + (Short.MIN_VALUE+1) + ",'foo']\n"
+ : "CLIENT PARALLEL " + saltBuckets + "-WAY RANGE SCAN OVER " + viewIndexPhysicalName + " [0," + (Short.MIN_VALUE+1) + ",'foo'] - ["+(saltBuckets.intValue()-1)+"," + (Long.MIN_VALUE+1) + ",'foo']\n"
+ + " [" + Long.toString(Short.MIN_VALUE + indexIdOffset) + ",'" + tenantId + "','f']\n" + " SERVER FILTER BY FIRST KEY ONLY",
+ ? "CLIENT PARALLEL 1-WAY RANGE SCAN OVER " + viewIndexPhysicalName + " [" + Short.MIN_VALUE + ",51]"
+ + Short.MIN_VALUE + ",51] - [" + (saltBuckets.intValue() - 1) + "," + Short.MIN_VALUE
-1 core tests. The patch failed these unit tests:
Test results: https://builds.apache.org/job/PreCommit-PHOENIX-Build/3103//testReport/
Release audit warnings: https://builds.apache.org/job/PreCommit-PHOENIX-Build/3103//artifact/patchprocess/patchReleaseAuditWarnings.txt
Console output: https://builds.apache.org/job/PreCommit-PHOENIX-Build/3103//console
This message is automatically generated.
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12985400/5559-4.x-HBase-1.5-v2.txt
against 4.x-HBase-1.5 branch at commit 70baf4c6805eb46420dd1fbaf71a093ab7d84a8b.
ATTACHMENT ID: 12985400
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 release audit. The applied patch generated 1 release audit warnings (more than the master's current 0 warnings).
-1 lineLengths. The patch introduces the following lines longer than 100:
+ "CLIENT PARALLEL 1-WAY RANGE SCAN OVER IDX" + tableName + " [" + Long.toString(Short.MIN_VALUE + expectedIndexIdOffset) + ",'" + tenantId + "','" + valuePrefix + "v2-1']\n"
+ "CLIENT PARALLEL 3-WAY RANGE SCAN OVER IDX" + tableName + " [0," + Long.toString(Short.MIN_VALUE + expectedIndexIdOffset) + ",'" + tenantId + "','" + valuePrefix +
+ "v2-1'] - ["+(saltBuckets.intValue()-1)+"," + Long.toString(Short.MIN_VALUE + expectedIndexIdOffset) + ",'" + tenantId + "','" + valuePrefix + "v2-1']\n"
+ ? "CLIENT PARALLEL 1-WAY RANGE SCAN OVER " + viewIndexPhysicalName +" [" + Short.MIN_VALUE + ",51]"
+ : "CLIENT PARALLEL " + saltBuckets + "-WAY RANGE SCAN OVER " + viewIndexPhysicalName + " [0," + Short.MIN_VALUE + ",51] - ["+(saltBuckets.intValue()-1)+"," + Short.MIN_VALUE + ",51]\nCLIENT MERGE SORT",
+ ? "CLIENT PARALLEL 1-WAY RANGE SCAN OVER " + viewIndexPhysicalName +" [" + (Short.MIN_VALUE+1) + ",'foo']\n"
+ : "CLIENT PARALLEL " + saltBuckets + "-WAY RANGE SCAN OVER " + viewIndexPhysicalName + " [0," + (Short.MIN_VALUE+1) + ",'foo'] - ["+(saltBuckets.intValue()-1)+"," + (Long.MIN_VALUE+1) + ",'foo']\n"
+ + " [" + Long.toString(Short.MIN_VALUE + indexIdOffset) + ",'" + tenantId + "','f']\n" + " SERVER FILTER BY FIRST KEY ONLY",
+ ? "CLIENT PARALLEL 1-WAY RANGE SCAN OVER " + viewIndexPhysicalName + " [" + Short.MIN_VALUE + ",51]"
+ + Short.MIN_VALUE + ",51] - [" + (saltBuckets.intValue() - 1) + "," + Short.MIN_VALUE
+1 core tests. The patch passed unit tests in .
Test results: https://builds.apache.org/job/PreCommit-PHOENIX-Build/3104//testReport/
Release audit warnings: https://builds.apache.org/job/PreCommit-PHOENIX-Build/3104//artifact/patchprocess/patchReleaseAuditWarnings.txt
Console output: https://builds.apache.org/job/PreCommit-PHOENIX-Build/3104//console
This message is automatically generated.
Test run looks good.
rajeshbabu please have a close look. I don't understand this enough to have great confidence in the change to BaseRegionObserver.
(I'll add the license header to the new tests)
Did some extra manual testing and all the problems I have seen are gone now.
(Including splitting while scanning and did not get any spurious stale boundary cache exceptions)
Fixed license header. Someone please have a look! It's time to get 4.15 out.
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12985580/5559-4.x-HBase-1.5-v3.txt
against 4.x-HBase-1.5 branch at commit dd5551b4e8cc17e5bd587e6c4f5f256a8212345a.
ATTACHMENT ID: 12985580
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 lineLengths. The patch introduces the following lines longer than 100:
+ "CLIENT PARALLEL 1-WAY RANGE SCAN OVER IDX" + tableName + " [" + Long.toString(Short.MIN_VALUE + expectedIndexIdOffset) + ",'" + tenantId + "','" + valuePrefix + "v2-1']\n"
+ "CLIENT PARALLEL 3-WAY RANGE SCAN OVER IDX" + tableName + " [0," + Long.toString(Short.MIN_VALUE + expectedIndexIdOffset) + ",'" + tenantId + "','" + valuePrefix +
+ "v2-1'] - ["+(saltBuckets.intValue()-1)+"," + Long.toString(Short.MIN_VALUE + expectedIndexIdOffset) + ",'" + tenantId + "','" + valuePrefix + "v2-1']\n"
+ ? "CLIENT PARALLEL 1-WAY RANGE SCAN OVER " + viewIndexPhysicalName +" [" + Short.MIN_VALUE + ",51]"
+ : "CLIENT PARALLEL " + saltBuckets + "-WAY RANGE SCAN OVER " + viewIndexPhysicalName + " [0," + Short.MIN_VALUE + ",51] - ["+(saltBuckets.intValue()-1)+"," + Short.MIN_VALUE + ",51]\nCLIENT MERGE SORT",
+ ? "CLIENT PARALLEL 1-WAY RANGE SCAN OVER " + viewIndexPhysicalName +" [" + (Short.MIN_VALUE+1) + ",'foo']\n"
+ : "CLIENT PARALLEL " + saltBuckets + "-WAY RANGE SCAN OVER " + viewIndexPhysicalName + " [0," + (Short.MIN_VALUE+1) + ",'foo'] - ["+(saltBuckets.intValue()-1)+"," + (Long.MIN_VALUE+1) + ",'foo']\n"
+ + " [" + Long.toString(Short.MIN_VALUE + indexIdOffset) + ",'" + tenantId + "','f']\n" + " SERVER FILTER BY FIRST KEY ONLY",
+ ? "CLIENT PARALLEL 1-WAY RANGE SCAN OVER " + viewIndexPhysicalName + " [" + Short.MIN_VALUE + ",51]"
+ + Short.MIN_VALUE + ",51] - [" + (saltBuckets.intValue() - 1) + "," + Short.MIN_VALUE
-1 core tests. The patch failed these unit tests:
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.UpgradeIT
Test results: https://builds.apache.org/job/PreCommit-PHOENIX-Build/3117//testReport/
Console output: https://builds.apache.org/job/PreCommit-PHOENIX-Build/3117//console
This message is automatically generated.
rajeshbabu seems to be MIA.
gjacoby, vincentpoon, kozdemir, or ckulkarni, could you have a look?
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12985675/5559-4.x-HBase-1.5-v5.txt
against 4.x-HBase-1.5 branch at commit f8cfa6b8b9f63ab080cdafa486bb4aabf043b1f8.
ATTACHMENT ID: 12985675
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 lineLengths. The patch introduces the following lines longer than 100:
+ "CLIENT PARALLEL 1-WAY RANGE SCAN OVER " + tableName + " [" + (1L + expectedIndexIdOffset) + ",'" + tenantId + "','" + valuePrefix + "v2-1']\n"
+ "CLIENT PARALLEL 3-WAY RANGE SCAN OVER " + tableName + " [" + (1L + expectedIndexIdOffset) + ",'" + tenantId + "','" + valuePrefix + "v2-1']\n"
+ "CLIENT PARALLEL 1-WAY RANGE SCAN OVER IDX" + tableName + " [" + (Short.MIN_VALUE + expectedIndexIdOffset) + ",'" + tenantId + "','" + valuePrefix + "v2-1']\n"
+ "CLIENT PARALLEL 3-WAY RANGE SCAN OVER IDX" + tableName + " [0," + (Short.MIN_VALUE + expectedIndexIdOffset) + ",'" + tenantId + "','" + valuePrefix +
+ "v2-1'] - ["+(saltBuckets.intValue()-1)+"," + (Short.MIN_VALUE + expectedIndexIdOffset) + ",'" + tenantId + "','" + valuePrefix + "v2-1']\n"
+ ? "CLIENT PARALLEL 1-WAY RANGE SCAN OVER " + viewIndexPhysicalName +" [" + Short.MIN_VALUE + ",51]"
+ : "CLIENT PARALLEL " + saltBuckets + "-WAY RANGE SCAN OVER " + viewIndexPhysicalName + " [0," + Short.MIN_VALUE + ",51] - ["+(saltBuckets.intValue()-1)+"," + Short.MIN_VALUE + ",51]\nCLIENT MERGE SORT",
+ ? "CLIENT PARALLEL 1-WAY RANGE SCAN OVER " + viewIndexPhysicalName +" [" + (Short.MIN_VALUE+1) + ",'foo']\n"
+ : "CLIENT PARALLEL " + saltBuckets + "-WAY RANGE SCAN OVER " + viewIndexPhysicalName + " [0," + (Short.MIN_VALUE+1) + ",'foo'] - ["+(saltBuckets.intValue()-1)+"," + (Short.MIN_VALUE+1) + ",'foo']\n"
+ + " [" + Long.toString(Short.MIN_VALUE + indexIdOffset) + ",'" + tenantId + "','f']\n" + " SERVER FILTER BY FIRST KEY ONLY",
+1 core tests. The patch passed unit tests in .
Test results: https://builds.apache.org/job/PreCommit-PHOENIX-Build/3118//testReport/
Console output: https://builds.apache.org/job/PreCommit-PHOENIX-Build/3118//console
This message is automatically generated.
-v5 is what has been reviewed on github... Going to commit tomorrow.
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12985675/5559-4.x-HBase-1.5-v5.txt
against 4.x-HBase-1.5 branch at commit f8cfa6b8b9f63ab080cdafa486bb4aabf043b1f8.
ATTACHMENT ID: 12985675
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 lineLengths. The patch introduces the following lines longer than 100:
+ "CLIENT PARALLEL 1-WAY RANGE SCAN OVER " + tableName + " [" + (1L + expectedIndexIdOffset) + ",'" + tenantId + "','" + valuePrefix + "v2-1']\n"
+ "CLIENT PARALLEL 3-WAY RANGE SCAN OVER " + tableName + " [" + (1L + expectedIndexIdOffset) + ",'" + tenantId + "','" + valuePrefix + "v2-1']\n"
+ "CLIENT PARALLEL 1-WAY RANGE SCAN OVER IDX" + tableName + " [" + (Short.MIN_VALUE + expectedIndexIdOffset) + ",'" + tenantId + "','" + valuePrefix + "v2-1']\n"
+ "CLIENT PARALLEL 3-WAY RANGE SCAN OVER IDX" + tableName + " [0," + (Short.MIN_VALUE + expectedIndexIdOffset) + ",'" + tenantId + "','" + valuePrefix +
+ "v2-1'] - ["+(saltBuckets.intValue()-1)+"," + (Short.MIN_VALUE + expectedIndexIdOffset) + ",'" + tenantId + "','" + valuePrefix + "v2-1']\n"
+ ? "CLIENT PARALLEL 1-WAY RANGE SCAN OVER " + viewIndexPhysicalName +" [" + Short.MIN_VALUE + ",51]"
+ : "CLIENT PARALLEL " + saltBuckets + "-WAY RANGE SCAN OVER " + viewIndexPhysicalName + " [0," + Short.MIN_VALUE + ",51] - ["+(saltBuckets.intValue()-1)+"," + Short.MIN_VALUE + ",51]\nCLIENT MERGE SORT",
+ ? "CLIENT PARALLEL 1-WAY RANGE SCAN OVER " + viewIndexPhysicalName +" [" + (Short.MIN_VALUE+1) + ",'foo']\n"
+ : "CLIENT PARALLEL " + saltBuckets + "-WAY RANGE SCAN OVER " + viewIndexPhysicalName + " [0," + (Short.MIN_VALUE+1) + ",'foo'] - ["+(saltBuckets.intValue()-1)+"," + (Short.MIN_VALUE+1) + ",'foo']\n"
+ + " [" + Long.toString(Short.MIN_VALUE + indexIdOffset) + ",'" + tenantId + "','f']\n" + " SERVER FILTER BY FIRST KEY ONLY",
+1 core tests. The patch passed unit tests in .
Test results: https://builds.apache.org/job/PreCommit-PHOENIX-Build/3119//testReport/
Console output: https://builds.apache.org/job/PreCommit-PHOENIX-Build/3119//console
This message is automatically generated.
FAILURE: Integrated in Jenkins build PreCommit-PHOENIX-Build #3122 (See https://builds.apache.org/job/PreCommit-PHOENIX-Build/3122/)
PHOENIX-5559 Fix remaining issues with Long viewIndexIds. (larsh: rev 910b72bf5d3b51a0c30ce43d9b19c0ce089cda62)
- (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/ViewIT.java
- (edit) phoenix-core/src/main/java/org/apache/phoenix/coprocessor/BaseScannerRegionObserver.java
- (edit) phoenix-core/src/main/java/org/apache/phoenix/util/MetaDataUtil.java
- (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/TenantSpecificViewIndexIT.java
- (edit) phoenix-core/src/main/java/org/apache/phoenix/iterate/ExplainTable.java
- (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/UpgradeIT.java
- (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/index/IndexUsageIT.java
- (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/index/LocalIndexIT.java
- (edit) phoenix-core/src/main/java/org/apache/phoenix/coprocessor/MetaDataEndpointImpl.java
- (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/BaseViewIT.java
- (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/index/MutableIndexFailureIT.java
- (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/index/GlobalIndexOptimizationIT.java
- (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/index/ViewIndexIT.java
- (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/BaseTenantSpecificViewIndexIT.java
- (add) phoenix-core/src/it/java/org/apache/phoenix/end2end/index/ShortViewIndexIdIT.java
- (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/index/ChildViewsUseParentViewIndexIT.java
SUCCESS: Integrated in Jenkins build Phoenix-4.x-HBase-1.5 #194 (See https://builds.apache.org/job/Phoenix-4.x-HBase-1.5/194/)
PHOENIX-5559 Fix remaining issues with Long viewIndexIds. (larsh: rev a16031441823bacc872ec8f832594d2280c6e83b)
- (add) phoenix-core/src/it/java/org/apache/phoenix/end2end/index/ShortViewIndexIdIT.java
- (edit) phoenix-core/src/main/java/org/apache/phoenix/iterate/ExplainTable.java
- (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/ViewIT.java
- (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/index/MutableIndexFailureIT.java
- (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/index/ViewIndexIT.java
- (edit) phoenix-core/src/main/java/org/apache/phoenix/coprocessor/BaseScannerRegionObserver.java
- (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/index/GlobalIndexOptimizationIT.java
- (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/TenantSpecificViewIndexIT.java
- (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/index/ChildViewsUseParentViewIndexIT.java
- (edit) phoenix-core/src/main/java/org/apache/phoenix/coprocessor/MetaDataEndpointImpl.java
- (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/BaseViewIT.java
- (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/UpgradeIT.java
- (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/index/LocalIndexIT.java
- (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/BaseTenantSpecificViewIndexIT.java
- (edit) phoenix-core/src/main/java/org/apache/phoenix/util/MetaDataUtil.java
- (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/index/IndexUsageIT.java
SUCCESS: Integrated in Jenkins build Phoenix-4.x-HBase-1.4 #309 (See https://builds.apache.org/job/Phoenix-4.x-HBase-1.4/309/)
PHOENIX-5559 Fix remaining issues with Long viewIndexIds. (larsh: rev 2c6986efccfa7d9bb323831334816ac6b9cb7d6d)
- (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/TenantSpecificViewIndexIT.java
- (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/UpgradeIT.java
- (edit) phoenix-core/src/main/java/org/apache/phoenix/coprocessor/MetaDataEndpointImpl.java
- (add) phoenix-core/src/it/java/org/apache/phoenix/end2end/index/ShortViewIndexIdIT.java
- (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/index/ViewIndexIT.java
- (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/index/GlobalIndexOptimizationIT.java
- (edit) phoenix-core/src/main/java/org/apache/phoenix/iterate/ExplainTable.java
- (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/BaseViewIT.java
- (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/index/LocalIndexIT.java
- (edit) phoenix-core/src/main/java/org/apache/phoenix/util/MetaDataUtil.java
- (edit) phoenix-core/src/main/java/org/apache/phoenix/coprocessor/BaseScannerRegionObserver.java
- (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/index/ChildViewsUseParentViewIndexIT.java
- (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/index/IndexUsageIT.java
- (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/BaseTenantSpecificViewIndexIT.java
- (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/ViewIT.java
- (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/index/MutableIndexFailureIT.java
FAILURE: Integrated in Jenkins build Phoenix-4.x-HBase-1.3 #594 (See https://builds.apache.org/job/Phoenix-4.x-HBase-1.3/594/)
PHOENIX-5559 Fix remaining issues with Long viewIndexIds. (larsh: rev 0c5ae7b3a64f2bec6b9bc08cb8fbe204eae9528b)
- (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/index/IndexUsageIT.java
- (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/index/LocalIndexIT.java
- (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/index/MutableIndexFailureIT.java
- (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/UpgradeIT.java
- (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/index/GlobalIndexOptimizationIT.java
- (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/BaseViewIT.java
- (add) phoenix-core/src/it/java/org/apache/phoenix/end2end/index/ShortViewIndexIdIT.java
- (edit) phoenix-core/src/main/java/org/apache/phoenix/util/MetaDataUtil.java
- (edit) phoenix-core/src/main/java/org/apache/phoenix/coprocessor/BaseScannerRegionObserver.java
- (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/TenantSpecificViewIndexIT.java
- (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/index/ViewIndexIT.java
- (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/index/ChildViewsUseParentViewIndexIT.java
- (edit) phoenix-core/src/main/java/org/apache/phoenix/iterate/ExplainTable.java
- (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/BaseTenantSpecificViewIndexIT.java
- (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/ViewIT.java
- (edit) phoenix-core/src/main/java/org/apache/phoenix/coprocessor/MetaDataEndpointImpl.java
stoty commented on PR #629:
URL: https://github.com/apache/phoenix/pull/629#issuecomment-1660487499
Already merged.
stoty closed pull request #629: PHOENIX-5559 Fix remaining issues with Long viewIndexIds.
URL: https://github.com/apache/phoenix/pull/629
The attached tests shows the problems of both
PHOENIX-5559(the data is written to the same index, and hence show up multiple times) andPHOENIX-5550(no data readable when an index is created on a data that contained data).I verified that both tests pass with phoenix.index.longViewIndex.enabled set to true.
m2je, FYI.
Edit: And this also covers the failure we've seen
PHOENIX-5486.If these two simple tests pass I'd feel much better.