Uploaded image for project: 'Samza'
  1. Samza
  2. SAMZA-1021

Remove the redundent poll waiting inside AsyncRunLoop blockIfBusy

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.11.0
    • Fix Version/s: 0.11.0
    • Component/s: None
    • Labels:
      None

      Description

      The waiting inside AsyncRunLoop is redundent given systemconsumers will block on polling for empty partition. Empty partitions happens when chooser returns null envelope and at the same time some task worker is avaliable to pick up messages. In this case we should rely on the systemconsumers.poll() to block instead of blocking in scheduling.

      The redundent waiting has been seen in our production job where both choose_ns and block_ns are high at the same time.

        Attachments

        1. SAMZA-1021.0.patch
          3 kB
          Xinyu Liu

          Activity

            People

            • Assignee:
              xinyu Xinyu Liu
              Reporter:
              xinyu Xinyu Liu
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: