Uploaded image for project: 'Phoenix'
  1. Phoenix
  2. PHOENIX-3853

Local Index - Writes to local index are twice as slow as global and get exponentially slower with PHOENIX-3827_v2 patch

Agile BoardAttach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 4.10.0
    • 4.11.0
    • None
    • HBase 1.3.1 4GB heap in local mode

    Description

      HBase 1.3.1 with head of Phoenix 4.x with/without PHOENIX-3827 v2 patch applied. This is with immutable non-covered local/global index on a single column with varying batch size when writing data to base table plus index.

      Batch Size Local Index with PHOENIX-3827_v2 patch (sec) Local Index without PHOENIX-3827_v2.patch (sec) Global (sec)
      100 0.02 0.03 0.013
      1000 0.3 0.3 0.13
      10000 4.3 2.6 1.3
      12500 8.1 3 1.6
      15000 13.3 3.1 1.9

      Schema and index

      CREATE TABLE IF NOT EXISTS T (OID CHAR(15) NOT NULL, PKP CHAR(3) NOT NULL, PIH CHAR(15) NOT NULL, FD DATE NOT NULL, SB CHAR(15) NOT NULL, BJ CHAR(15), JR VARCHAR, FIELD VARCHAR, YM VARCHAR, WN VARCHAR, LG VARCHAR, XHJ VARCHAR, HF VARCHAR, GA VARCHAR, MX VARCHAR, NZ DECIMAL, JV DECIMAL, AG DATE, KV DATE, JK VARCHAR, DK VARCHAR, EU DATE, OE VARCHAR, DV INTEGER, IK VARCHAR CONSTRAINT PK PRIMARY KEY ( OID, PKP, PIH, FD DESC, SB )) VERSIONS=1,IMMUTABLE_ROWS=true
      
      CREATE INDEX IF NOT EXISTS IDXT ON T (JV)
      

      Data CSV
      https://expirebox.com/download/1cea73af1831b5193f0539d6e3442292.html

      Rajeshbabu Chintaguntla, [~lhofhansl], James R. Taylor

      Attachments

        1. after_3858.png
          31 kB
          Mujtaba Chohan
        2. PHOENIX-3853_test_only.patch
          5 kB
          James R. Taylor
        3. batch_mutable.png
          22 kB
          Mujtaba Chohan
        4. batch_time.png
          24 kB
          Mujtaba Chohan

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            rajeshbabu Rajeshbabu Chintaguntla
            mujtabachohan Mujtaba Chohan
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment