Uploaded image for project: 'Apache Storm'
  1. Apache Storm
  2. STORM-789

Enhance topology context sent to multi-lang bolts and spouts

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.10.0
    • Component/s: storm-multilang
    • Labels:
      None

      Description

      I'm about to submit a pull request that adds a lot more contextual information to the "context" JSON dictionary that gets sent to multi-lang bolts and spouts as part of their initial handshake. These additions will make is so non-JVM developers have access to the sources, targets, and field names for their bolts and spouts.

      We will add support for this in streamparse (one of the more popular Python libraries for Storm) as soon as this gets merged in.

      Here are the details of what I've added:

      • componentid: the component ID for this task, so you don't have to look it up
        in "task->component" by "taskid"
      • streams: a list of all of the streams for this task
      • stream->outputfields: a mapping from stream names to output fields for that
        stream
      • stream->target->grouping: a mapping from stream names to the targets for this
        task to the kind of grouping they use.
      • sources->grouping: a mapping from the component IDs of the sources to their
        grouping.

      Note on groupings: groupings are either represented as a string (e.g., "SHUFFLE") or a list of strings for field groupings.

        Attachments

          Activity

            People

            • Assignee:
              dan.blanchard Dan Blanchard
              Reporter:
              dan.blanchard Dan Blanchard
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: