Kafka
  1. Kafka
  2. KAFKA-48

Implement optional "long poll" support in fetch request

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      Currently, the fetch request is non-blocking. If there is nothing on the broker for the consumer to retrieve, the broker simply returns an empty set to the consumer. This can be inefficient, if you want to ensure low-latency because you keep polling over and over. We should make a blocking version of the fetch request so that the fetch request is not returned until the broker has at least one message for the fetcher or some timeout passes.

      1. KAFKA-48-v4.patch
        62 kB
        Jay Kreps
      2. kafka-48-v3-to-v4-changes.diff
        8 kB
        Jay Kreps
      3. KAFKA-48-v3.patch
        61 kB
        Jay Kreps
      4. KAFKA-48-v2.patch
        46 kB
        Jay Kreps
      5. KAFKA-48.patch
        30 kB
        Jay Kreps

        Issue Links

          Activity

          Jay Kreps made changes -
          Status Reopened [ 4 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          Jay Kreps made changes -
          Attachment kafka-48-v3-to-v4-changes.diff [ 12522017 ]
          Jay Kreps made changes -
          Attachment KAFKA-48-v4.patch [ 12522016 ]
          Jay Kreps made changes -
          Attachment KAFKA-48-v3.patch [ 12521358 ]
          Jay Kreps made changes -
          Attachment KAFKA-48-v2.patch [ 12521353 ]
          Alan Cabrera made changes -
          Reporter Alan Cabrera [ maguro ] Jun Rao [ junrao ]
          Jay Kreps made changes -
          Attachment KAFKA-48.patch [ 12513197 ]
          Jay Kreps made changes -
          Attachment KAFKA-48-socket-server-refactor-draft.patch [ 12501564 ]
          Jay Kreps made changes -
          Link This issue is blocked by KAFKA-202 [ KAFKA-202 ]
          Jay Kreps made changes -
          Assignee Jay Kreps [ jkreps ]
          Jay Kreps made changes -
          Attachment KAFKA-48-socket-server-refactor-draft.patch [ 12501564 ]
          Jay Kreps made changes -
          Description Currently, the fetch request is non-blocking. If there is nothing on the broker for the consumer to retrieve, the broker simply returns an empty set to the consumer. This can be inefficient. We should make a blocking version of the fetcher so that the fetch request is not returned until the broker has at least one message for the fetcher. Currently, the fetch request is non-blocking. If there is nothing on the broker for the consumer to retrieve, the broker simply returns an empty set to the consumer. This can be inefficient, if you want to ensure low-latency because you keep polling over and over. We should make a blocking version of the fetch request so that the fetch request is not returned until the broker has at least one message for the fetcher or some timeout passes.
          Jay Kreps made changes -
          Summary Make fetcher blocking Implement optional "long poll" support in fetch request
          Alan Cabrera made changes -
          Workflow jira [ 12621001 ] no-reopen-closed, patch-avail [ 12626245 ]
          Alan Cabrera made changes -
          Resolution Unresolved [ 9 ]
          Status Closed [ 6 ] Reopened [ 4 ]
          Alan Cabrera made changes -
          Status Open [ 1 ] Closed [ 6 ]
          Alan Cabrera made changes -
          Summary KAFKA-23 Make fetcher blocking Make fetcher blocking
          Issue Type Sub-task [ 7 ] Bug [ 1 ]
          Reporter Alan Cabrera [ maguro ]
          Jun Rao made changes -
          Link This issue blocks KAFKA-50 [ KAFKA-50 ]
          Jun Rao made changes -
          Link This issue is blocked by KAFKA-50 [ KAFKA-50 ]
          Jun Rao made changes -
          Field Original Value New Value
          Link This issue is blocked by KAFKA-50 [ KAFKA-50 ]
          Anonymous created issue -

            People

            • Assignee:
              Jay Kreps
              Reporter:
              Jun Rao
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development