[SENTRY-1931] NameNode only gets full snapshot once

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.0.0
• Fix Version/s: 2.0.0
• Component/s: Sentry
• Assignee: Sergio Peña
• Reporter: Alex Kolbasov
• Created: 07/Sep/17 21:37
• Updated: 08/Sep/17 01:18
• Resolved: 08/Sep/17 01:18

Description

SENTRY-1815 introduced the notion of ImageID, but it turns out that it doesn't work quite as expected.

The problem is that when we send partial update the image number isn't set:

  public List<PathsUpdate> retrieveDelta(long seqNum) throws Exception {
      ...
      List<PathsUpdate> updates = new ArrayList<>(mSentryPathChanges.size());
      for (MSentryPathChange mSentryPathChange : mSentryPathChanges) {
        // Gets the changeID from the persisted MSentryPathChange.
        long changeID = mSentryPathChange.getChangeID();
        // Creates a corresponding PathsUpdate and deserialize the
        // persisted delta update in JSON format to TPathsUpdate with
        // associated changeID.
        PathsUpdate pathsUpdate = new PathsUpdate();
        pathsUpdate.JSONDeserialize(mSentryPathChange.getPathChange());
        pathsUpdate.setSeqNum(changeID);
        updates.add(pathsUpdate);
      }
      return updates;
    }
  }

When it isn't set, thrift uses -1 as the default (as specified in our Thrift file).

The code in UpdateableAuthzPaths.updatePartial does this:

  @Override
  public void updatePartial(Iterable<PathsUpdate> updates, ReadWriteLock lock) {
      for (PathsUpdate update : updates) {
        applyPartialUpdate(update);
        seqNum.set(update.getSeqNum());
        imgNum.set(update.getImgNum()); // Here
      }
  }

so we set imgNum to -1 when we receive any partial update and then we always request for all updates from -1 and never receive a full update.

Issue Links

• relates to: SENTRY-1815 Send new HMS snapshots to HDFS requesting an old generation ID
• links to: Review Board