Crunch
  1. Crunch
  2. CRUNCH-90

Object reuse is not accounted for in mapper fusion

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.4.0
    • Component/s: None
    • Labels:
      None

      Description

      When multiple DoFns are run over the same output (i.e. in the case of mapper fusion), the same value object is passed to multiple underlying DoFns. If the state of that value object is changed by one DoFn, other DoFns are called with the updated object.

      This is a situation that can happen quite easily when the input of a DoFn is simply updated and then emitted. In general, this bug will only affect values whose type is the same as the underlying serialization type.

      1. CRUNCH-90.patch
        12 kB
        Gabriel Reid
      2. CRUNCH-90-reflect.patch
        48 kB
        Josh Wills
      3. CRUNCH-90-reflect.patch
        74 kB
        Gabriel Reid

        Activity

          People

          • Assignee:
            Gabriel Reid
            Reporter:
            Gabriel Reid
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development