Uploaded image for project: 'Phoenix'
  1. Phoenix
  2. PHOENIX-3271

Distribute UPSERT SELECT across cluster

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 4.10.0
    • None
    • None

    Description

      Based on some informal testing we've done, it seems that creation of a local index is orders of magnitude faster that creation of global indexes (17 seconds versus 10-20 minutes - though more data is written in the global index case). Under the covers, a global index is created through the running of an UPSERT SELECT. Also, UPSERT SELECT provides an easy way of copying a table. In both of these cases, the data being upserted must all flow back to the same client which can become a bottleneck for a large table. Instead, what can be done is to push each separate, chunked UPSERT SELECT call out to a different region server for execution there. One way we could implement this would be to have an endpoint coprocessor push the chunked UPSERT SELECT out to each region server and return the number of rows that were upserted back to the client.

      Attachments

        1. PHOENIX-3271.patch
          15 kB
          Ankit Singhal
        2. PHOENIX-3271_v1.patch
          16 kB
          Ankit Singhal
        3. PHOENIX-3271_v2.patch
          21 kB
          Ankit Singhal
        4. PHOENIX-3271_v3.patch
          21 kB
          Ankit Singhal
        5. PHOENIX-3271_v4.patch
          22 kB
          Ankit Singhal
        6. PHOENIX-3271_v5.patch
          27 kB
          Ankit Singhal
        7. PHOENIX-3271_v5_rebased.patch
          26 kB
          Ankit Singhal

        Activity

          People

            ankit@apache.org Ankit Singhal
            jamestaylor James R. Taylor
            Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: