Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-4711

[Plasma] enhance plasma client interfaces to work with multiple objects

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: 2.0.0
    • Component/s: C++ - Plasma
    • Labels:
      None

      Description

      Right now the Delete() interface in plasma client supports deleting multiple objects in a single shot, so that we can save IPCs (inter-processing communication) between plasma clients and plasma store. This reduces latency for plasma clients, and also improve the actual throughput for plasma store.

      I made a simple prototype for changing Release() function as well, when batching release 10 objects in a single IPC, it only takes about 1/6 of the time compared with using 10 separate IPCs. Also from profiling, processing IPCs takes a lot of CPU cycles in plasma store currently, as UNIX domain socket processing needs to go through kernel, thus batching multiple IPCs into a single one should greatly improve the plasma store performance as well.

       

      This change mostly applies to Release(), Seal(), and Create() interfaces.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              zhijunfu Zhijun Fu
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: