Uploaded image for project: 'Samza'
  1. Samza
  2. SAMZA-1516

Another round of issues found by BEAM tests

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.14.0
    • Fix Version/s: 0.14.0
    • Component/s: None
    • Labels:
      None

      Description

      A couple of more fixes:

      1. fix a bug of identifying input streams for an operator. The logic is inside the traversal of operator graph inside OperatorImplGraph. Basically we need to traverse further to add the input stream to the operator even the operator is already created.

      2. for partitionBy, right now if there is null key and null partition key, Kafka will do round-robin (https://kafka.apache.org/0102/javadoc/org/apache/kafka/clients/producer/ProducerRecord.html). This is causing problems for BEAM reduce. Since BEAM assume records without key will go to a single reducer. Add a fix in partitonBy to set the partitionKey.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                xinyu Xinyu Liu
                Reporter:
                xinyu Xinyu Liu
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: