Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-2045

[C++][Plasma] More primitive operations on plasma store

    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: 2.0.0
    • Component/s: C++ - Plasma
    • Labels:
      None

      Description

      Hi Developers,

      I found plasma store very useful – it's fast and simple to use. However, I think there are more operations that can make it a more general IPC/messaging tool and potentially helpful in more scenarios.

      Conceptually, an object store can support the following "put" methods:

      1. Evict when full
      2. Wait for space when full, perhaps with a timeout (i.e. blocking)
      3. Return failure when full (i.e. non-blocking)

      And the following "get" methods:

      1. Wait for the object to appear (i.e. blocking)
      2. Return failure when object doesn't exist (i.e. non-blocking)
      3. Remove the object after get

      Some of the above features can be implemented with others. But some of them are primitives (e.g. return failure when full) that needs to be supported.

       

      My use case: I wanted to use plasma to send/recv large buffers between processes, i.e. build a message passing interface on top of shared memory. Plasma has made it quite easy (only have to send/recv the id) and efficient (faster than unix pipe). But "evict when full" is now the only available "put" method, so that could create many trouble if I want to ensure message delivery.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              ppwwyyxx Yuxin Wu
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: