Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-2928 YARN Timeline Service v.2: alpha 1
  3. YARN-3134

[Storage implementation] Exploiting the option of using Phoenix to access HBase backend

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 2.9.0, 3.0.0-alpha1
    • timelineserver
    • None
    • Reviewed

    Description

      Quote the introduction on Phoenix web page:

      Apache Phoenix is a relational database layer over HBase delivered as a client-embedded JDBC driver targeting low latency queries over HBase data. Apache Phoenix takes your SQL query, compiles it into a series of HBase scans, and orchestrates the running of those scans to produce regular JDBC result sets. The table metadata is stored in an HBase table and versioned, such that snapshot queries over prior versions will automatically use the correct schema. Direct use of the HBase API, along with coprocessors and custom filters, results in performance on the order of milliseconds for small queries, or seconds for tens of millions of rows.

      It may simply our implementation read/write data from/to HBase, and can easily build index and compose complex query.

      Attachments

        1. YARN-3134DataSchema.pdf
          185 kB
          Li Lu
        2. YARN-3134-040915_poc.patch
          20 kB
          Li Lu
        3. YARN-3134-041015_poc.patch
          26 kB
          Li Lu
        4. YARN-3134-041415_poc.patch
          34 kB
          Li Lu
        5. YARN-3134-042115.patch
          34 kB
          Li Lu
        6. YARN-3134-042715.patch
          36 kB
          Li Lu
        7. SettingupPhoenixstorageforatimelinev2end-to-endtest.pdf
          128 kB
          Li Lu
        8. YARN-3134-YARN-2928.001.patch
          36 kB
          Li Lu
        9. YARN-3134-YARN-2928.002.patch
          39 kB
          Li Lu
        10. YARN-3134-YARN-2928.003.patch
          40 kB
          Li Lu
        11. YARN-3134-YARN-2928.004.patch
          40 kB
          Li Lu
        12. YARN-3134-YARN-2928.005.patch
          41 kB
          Li Lu
        13. YARN-3134-YARN-2928.006.patch
          39 kB
          Li Lu
        14. hadoop-zshen-nodemanager-d-128-95-184-84.dhcp4.washington.edu.out
          307 kB
          Zhijie Shen
        15. YARN-3134-YARN-2928.007.patch
          38 kB
          Li Lu

        Issue Links

          blocks

          Sub-task - The sub-task of the issue YARN-3529 Add miniHBase cluster and Phoenix support to ATS v2 unit tests

          • Major - Major loss of function.
          • Resolved
          is blocked by

          Sub-task - The sub-task of the issue YARN-3551 Consolidate data model change according to the backend implementation

          • Major - Major loss of function.
          • Resolved
          is related to

          Sub-task - The sub-task of the issue YARN-3411 [Storage implementation] explore & create the native HBase schema for writes

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved

          Sub-task - The sub-task of the issue YARN-3595 Performance optimization using connection cache of Phoenix timeline writer

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              gtcarrera9 Li Lu
              zjshen Zhijie Shen
              Votes:
              0 Vote for this issue
              Watchers:
              15 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: