[HBASE-19423] Replication entries are not filtered correctly when replication scope is set through WAL Co-processor

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Incomplete
• Labels: Replication WAL
• Assignee: Unassigned
• Reporter: Mohammad Arshad
• Created: 04/Dec/17 22:07
• Updated: 14/Dec/18 00:33
• Resolved: 14/Dec/18 00:33

Description

Replicaion scope set in WALObserver is getting reset in Replication.scopeWALEdits().
Because of this problem custom implementation of WALObserver can not be used as a replication filter.
Suppose WALObserver implementation has logic to filter all entries from family f2

// Filter all family f2 rows
  public static class ReplicationFilterWALCoprocessor extends BaseWALObserver {
    @Override
    public boolean preWALWrite(ObserverContext<? extends WALCoprocessorEnvironment> ctx,
        HRegionInfo info, WALKey logKey, WALEdit logEdit) throws IOException {
      ArrayList<Cell> cells = logEdit.getCells();
      for (Cell cell : cells) {
        byte[] fam = CellUtil.cloneFamily(cell);
        if ("f2".equals(Bytes.toString(fam))) {
          NavigableMap<byte[], Integer> scopes = logKey.getScopes();
          if (scopes == null) {
            logKey.setScopes(new TreeMap<byte[], Integer>(Bytes.BYTES_COMPARATOR));
          }
          logKey.getScopes().put(fam, HConstants.REPLICATION_SCOPE_LOCAL);
        }
      }
      return false;
    }
  }

This logic can not work as org.apache.hadoop.hbase.replication.regionserver.Replication.scopeWALEdits() recreates and populates scopes.
SOLUTION:
In Replication.scopeWALEdits(), create scopes map only if WALKey does not have it.

NavigableMap<byte[], Integer> scopes = logKey.getScopes();
    if (scopes == null) {
      scopes = new TreeMap<byte[], Integer>(Bytes.BYTES_COMPARATOR);
    }