[PHOENIX-3853] Local Index - Writes to local index are twice as slow as global and get exponentially slower with PHOENIX-3827_v2 patch

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 4.10.0
• Fix Version/s: 4.11.0
• Labels: localIndex

Environment

HBase 1.3.1 4GB heap in local mode

• Assignee: Rajeshbabu Chintaguntla
• Reporter: Mujtaba Chohan
• Created: 15/May/17 22:23
• Updated: 10/Jun/17 02:08
• Resolved: 24/May/17 09:38

Description

HBase 1.3.1 with head of Phoenix 4.x with/without PHOENIX-3827 v2 patch applied. This is with immutable non-covered local/global index on a single column with varying batch size when writing data to base table plus index.

Batch Size	Local Index with PHOENIX-3827_v2 patch (sec)	Local Index without PHOENIX-3827_v2.patch (sec)	Global (sec)
100	0.02	0.03	0.013
1000	0.3	0.3	0.13
10000	4.3	2.6	1.3
12500	8.1	3	1.6
15000	13.3	3.1	1.9

Schema and index

CREATE TABLE IF NOT EXISTS T (OID CHAR(15) NOT NULL, PKP CHAR(3) NOT NULL, PIH CHAR(15) NOT NULL, FD DATE NOT NULL, SB CHAR(15) NOT NULL, BJ CHAR(15), JR VARCHAR, FIELD VARCHAR, YM VARCHAR, WN VARCHAR, LG VARCHAR, XHJ VARCHAR, HF VARCHAR, GA VARCHAR, MX VARCHAR, NZ DECIMAL, JV DECIMAL, AG DATE, KV DATE, JK VARCHAR, DK VARCHAR, EU DATE, OE VARCHAR, DV INTEGER, IK VARCHAR CONSTRAINT PK PRIMARY KEY ( OID, PKP, PIH, FD DESC, SB )) VERSIONS=1,IMMUTABLE_ROWS=true

CREATE INDEX IF NOT EXISTS IDXT ON T (JV)

Data CSV
https://expirebox.com/download/1cea73af1831b5193f0539d6e3442292.html

rajeshbabu, lhofhansl, jamestaylor

Issue Links

• is related to: PHOENIX-3827 Make use of HBASE-15600 to write local index mutations along with data mutations atomically
• relates to: PHOENIX-3858 Index maintenance not required for local indexes of table with immutable rows