River
  1. River
  2. RIVER-324

Under certain circumstances, the ServiceDiscoveryManager internal LookupCache implementation can incorrectly process attribute change events before the lookup snapshot is processed.

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: River_2.1.1
    • Fix Version/s: River_2.1.2
    • Component/s: net_jini_lookup
    • Labels:
      None

      Description

      When an attribute change event is received from the
      lookup service between the time the cache registers
      the event listener and the initial LookupTask takes
      the snapshot of the associated service state, the
      change event can get processed first, which can
      result in incorrect attribute state.

      This bug has been observed in a currently deployed
      system, generally at startup when the services of
      the system are changing their attributes from an
      initial, 'unknown' state, to a discovered state
      that is shared among those services. What has been
      observed is a sequence like the following:

      1. event registration is sent to the lookup service

      2. snapshot is requested (LookupTask is queued)

      3. the lookup service sends back in the requested
      snapshot, the initial state the service registered
      for itself

      4. the service sends an attribute modification
      request to the lookup service, which sends an
      attribute change event to the cache

      5. before the cache's LookupTask processes the
      snapshot from the lookup service, the event
      arrives and the event processing thread of the
      cache processes the event containing the latest
      state of the service's attributes.

      6. the cache then processes the snapshot, replacing
      the latest, most up-to-date attribute state with
      the original, initial state reflected in the
      snapshot.

      7. the cache now has an incorrect view of the
      service's state.

      Bob Scheifler has implemented a simple fix; which
      is (quoting Bob), "to have the LookupTask execute
      the tasks it creates directly, rather than queueing
      them." That is, force any pending snapshot processing
      tasks to be executed before the event processing
      tasks.

      Note that with the proposed fix, if more than
      one lookup service is running, it is possible for
      an attribute to "regress" as the lookup services
      do not receive a given attribute change at exactly
      the same time, but the inconsistency will eventually
      correct itself as the cache receives each attribute
      change event, and so should not be a permanent
      condition.

      1. river-324.patch
        4 kB
        Brian Murphy
      2. river-324-2.diff
        0.5 kB
        Bob Scheifler

        Activity

        Peter Firmstone made changes -
        Status Reopened [ 4 ] Closed [ 6 ]
        Resolution Fixed [ 1 ]
        Hide
        Peter Firmstone added a comment -

        Fix to fix has been committed to trunk, new Apache River 2.1.2 Release Artefacts, will be built and re submitted to a [VOTE] on the weekend.

        Show
        Peter Firmstone added a comment - Fix to fix has been committed to trunk, new Apache River 2.1.2 Release Artefacts, will be built and re submitted to a [VOTE] on the weekend.
        Bob Scheifler made changes -
        Resolution Fixed [ 1 ]
        Status Closed [ 6 ] Reopened [ 4 ]
        Assignee Brian Murphy [ btmurphy ]
        Hide
        Bob Scheifler added a comment -

        Original fix had a nasty flaw. Fix to fix has been attached.

        Show
        Bob Scheifler added a comment - Original fix had a nasty flaw. Fix to fix has been attached.
        Bob Scheifler made changes -
        Attachment river-324-2.diff [ 12436543 ]
        Hide
        Bob Scheifler added a comment -

        Fix to fix.

        Show
        Bob Scheifler added a comment - Fix to fix.
        Peter Firmstone made changes -
        Status Open [ 1 ] Closed [ 6 ]
        Resolution Fixed [ 1 ]
        Hide
        Peter Firmstone added a comment -

        Patch file has been integrated into the build, tests haven't uncovered any remaining issues, if any issues are uncovered / reported by downstream users, this item should be reopened.

        Show
        Peter Firmstone added a comment - Patch file has been integrated into the build, tests haven't uncovered any remaining issues, if any issues are uncovered / reported by downstream users, this item should be reopened.
        Hide
        Brian Murphy added a comment -

        Sure. Although if anyone runs into problems that
        the tests haven't revealed, we'll need to re-open
        it and investigate.

        Brian

        Show
        Brian Murphy added a comment - Sure. Although if anyone runs into problems that the tests haven't revealed, we'll need to re-open it and investigate. Brian
        Hide
        Peter Firmstone added a comment -

        Can this issue be safely closed?

        Show
        Peter Firmstone added a comment - Can this issue be safely closed?
        Hide
        Brian Murphy added a comment -

        Changeset 833922: contains the proposed fix reflected
        in the attached patch file, and described in this issue's
        description.

        Verification: ran all ServiceDiscoveryManager tests and
        observed no errors or problems.

        btm-08-Nov-2009

        Show
        Brian Murphy added a comment - Changeset 833922: contains the proposed fix reflected in the attached patch file, and described in this issue's description. Verification: ran all ServiceDiscoveryManager tests and observed no errors or problems. btm-08-Nov-2009
        Brian Murphy made changes -
        Field Original Value New Value
        Attachment river-324.patch [ 12424323 ]
        Hide
        Brian Murphy added a comment -

        Attached patch file contains the diffs that reflect the proposed
        fix referenced in this issue's description.

        Show
        Brian Murphy added a comment - Attached patch file contains the diffs that reflect the proposed fix referenced in this issue's description.
        Brian Murphy created issue -

          People

          • Assignee:
            Unassigned
            Reporter:
            Brian Murphy
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development