Kafka
  1. Kafka
  2. KAFKA-564

Wildcard-based topic consumption should assign partitions to threads uniformly

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Duplicate
    • Affects Version/s: 0.8.1
    • Fix Version/s: 0.9.0
    • Component/s: None
    • Labels:
      None

      Description

      Right now, if when a client uses createMessageStreamsByFilter and specifies 'n' streams (threads), 'n' should be <= the max partition count of any topic. If it is greater than that, the excess threads will be idle. However, it would be better to allow a greater number of threads and spread all the available partitions across the threads.
      This should not be too difficult, but may require significant refactoring.

      Although it is relevant to current trunk/0.7, we will target this for post-0.8.

        Issue Links

          Activity

          Joel Koshy made changes -
          Status Open [ 1 ] Resolved [ 5 ]
          Resolution Duplicate [ 3 ]
          Joel Koshy made changes -
          Link This issue duplicates KAFKA-687 [ KAFKA-687 ]
          Neha Narkhede made changes -
          Fix Version/s 0.9.0 [ 12323928 ]
          Fix Version/s 0.8.1 [ 12322960 ]
          Joel Koshy made changes -
          Field Original Value New Value
          Description This should not be too difficult, but may require significant refactoring.

          Although it is relevant to current trunk/0.7, we will target this for post-0.8.
          Right now, if when a client uses createMessageStreamsByFilter and specifies 'n' streams (threads), 'n' should be <= the max partition count of any topic. If it is greater than that, the excess threads will be idle. However, it would be better to allow a greater number of threads and spread all the available partitions across the threads.
          This should not be too difficult, but may require significant refactoring.

          Although it is relevant to current trunk/0.7, we will target this for post-0.8.
          Joel Koshy created issue -

            People

            • Assignee:
              Unassigned
              Reporter:
              Joel Koshy
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development