Uploaded image for project: 'Giraph (Retired)'
  1. Giraph (Retired)
  2. GIRAPH-1027

Improve flow-control on sender side

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 1.2.0
    • graph
    • None

    Description

      Currently, a sender worker will keep all open requests (and optionally up to a certain number of total open requests) in its own memory. This behavior may cause high memory usage in sender side. Also, since messages can arrive to a worker at an arbitrary rate, receiver may not have the ability to handle all incoming messages, hence we may see a large memory footprint in receiver as well. This diff addresses the problem by limiting the number of open requests per worker in sender side. Also, it provides a cache of unsent requests on sender in case the sender already sent enough messages to another worker but has not received any response back.

      Attachments

        Activity

          People

            heslami Hassan Eslami
            heslami Hassan Eslami
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: