Uploaded image for project: 'Samza'
  1. Samza
  2. SAMZA-1391

System generated intermediate streams should have robust naming scheme

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Duplicate
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      In certain class of high level stream applications, Samza takes care of creating intermediate streams. Currently, we use the combination of job name, job id and the operator id to come up with the logical and physical stream name. We need to come up with a unique, robust naming scheme to prevent collision. We need to either use a unique name or detect the collision and perform necessary purge/cleanup steps prior to stream creation.

      An example where this bug is exploited is...
      Consider a stream application with following phases.

      input --> partition --> output

      If the input source were to change or the user provided partition function changes, our DAG would still be the same resulting in the same logical and physical stream name.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                bharathkk Bharath Kumarasubramanian
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: