Uploaded image for project: 'Crunch'
  1. Crunch
  2. CRUNCH-90

Object reuse is not accounted for in mapper fusion

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.4.0
    • Component/s: None
    • Labels:
      None

      Description

      When multiple DoFns are run over the same output (i.e. in the case of mapper fusion), the same value object is passed to multiple underlying DoFns. If the state of that value object is changed by one DoFn, other DoFns are called with the updated object.

      This is a situation that can happen quite easily when the input of a DoFn is simply updated and then emitted. In general, this bug will only affect values whose type is the same as the underlying serialization type.

        Attachments

        1. CRUNCH-90-reflect.patch
          74 kB
          Gabriel Reid
        2. CRUNCH-90-reflect.patch
          48 kB
          Josh Wills
        3. CRUNCH-90.patch
          12 kB
          Gabriel Reid

          Activity

            People

            • Assignee:
              gabriel.reid Gabriel Reid
              Reporter:
              gabriel.reid Gabriel Reid
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: