Uploaded image for project: 'Phoenix'
  1. Phoenix
  2. PHOENIX-2995

Write performance severely degrades with large number of views

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 4.9.0, 4.8.1
    • Labels:

      Description

      Write performance for each 1K batch degrades significantly when there are 10K views being written in random with default phoenix.client.maxMetaDataCacheSize. With all views created, upsert rate remains around 25 seconds per 1K batch i.e. ~2K rows/min upsert rate.

      When phoenix.client.maxMetaDataCacheSize is increased to 100MB+ then view does not need to get re-resolved and upsert rate gets back to normal ~60K rows/min.

      With 100K views and phoenix.client.maxMetaDataCacheSize set to 1GB, I wasn't able create all 100K views as upsert time for each 1K batch keeps on steadily increasing.

      Following graph shows 1K batch upsert rate over time with variation of number of views. Rows are upserted to random views CREATE VIEW IF NOT EXISTS ... APPEND_ONLY_SCHEMA = true, UPDATE_CACHE_FREQUENCY=900000 is executed before upsert statement.

      Base table is also created with APPEND_ONLY_SCHEMA = true, UPDATE_CACHE_FREQUENCY = 900000, AUTO_PARTITION_SEQ

        Attachments

        1. create_view_and_upsert.png
          19 kB
          Mujtaba Chohan
        2. image.png
          23 kB
          Thomas D'Silva
        3. image2.png
          28 kB
          Thomas D'Silva
        4. image3.png
          14 kB
          Thomas D'Silva
        5. PHOENIX-2995.patch
          10 kB
          Thomas D'Silva
        6. PHOENIX-2995-v2.patch
          32 kB
          Thomas D'Silva
        7. PHOENIX-2995-v3.patch
          95 kB
          Thomas D'Silva
        8. PHOENIX-2995-v4.patch
          87 kB
          Thomas D'Silva
        9. PHOENIX-2995-v5.patch
          82 kB
          Thomas D'Silva
        10. PHOENIX-2995-v6.patch
          87 kB
          Thomas D'Silva
        11. upsert_rate.png
          35 kB
          Mujtaba Chohan

          Issue Links

            Activity

              People

              • Assignee:
                tdsilva Thomas D'Silva
                Reporter:
                mujtabachohan Mujtaba Chohan
              • Votes:
                0 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: