Uploaded image for project: 'S2Graph'
  1. S2Graph
  2. S2GRAPH-26

Apply squash optimization on mutateElements and make it default behavior on Storage.

    Details

    • Type: Improvement
    • Status: In Progress
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None

      Description

      Currently AsynchbaseStorage that implement Storage trait using HBase use squash optimization on snapshot edges at mutateEdges.

      for example, if there are requests on same snapshotEdge that consists of (insert, delete, insert, delete, insert) in their timestamp order, then we only need to apply last insert.

      ex) lets assume that there are 5 requests on from “shon” to “dun” with label “friend”, insert(t0), delete(t1), insert(t2), delete(t3), insert(t4).

      without squashing same snapshot edges in memory, then 5 of following actions need to be done.

      1. fetch snapshot edge
      2. lock this snapshot edge.
      3. build new update and delete, insert, degree on IndexEdges.
      4. fire above update/delete/insert/degree mutations into HBase.
      5. release lock on this snapshotEdge.

      we can do 1, 2, 5(fetch, lock, release lock) one time and squash mutations that built from multiple requests and squash them.

      above logic needs to keep consistency between multiple indexEdges and purpose above logic is make only one thread on same snapshot edge be able to mutate(note the lock). since we will acquire lock on this edge, it is much efficient squash multiple requests on same snapshotEdge to avoid heavy operation, fetch, lock, release lock.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                daewon Daewon Jeong
                Reporter:
                steamshon DOYUNG YOON
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Due:
                  Created:
                  Updated:

                  Time Tracking

                  Estimated:
                  Original Estimate - 72h
                  72h
                  Remaining:
                  Remaining Estimate - 72h
                  72h
                  Logged:
                  Time Spent - Not Specified
                  Not Specified