Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-19423

Replication entries are not filtered correctly when replication scope is set through WAL Co-processor

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Incomplete
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:

      Description

      Replicaion scope set in WALObserver is getting reset in Replication.scopeWALEdits().
      Because of this problem custom implementation of WALObserver can not be used as a replication filter.
      Suppose WALObserver implementation has logic to filter all entries from family f2

      // Filter all family f2 rows
        public static class ReplicationFilterWALCoprocessor extends BaseWALObserver {
          @Override
          public boolean preWALWrite(ObserverContext<? extends WALCoprocessorEnvironment> ctx,
              HRegionInfo info, WALKey logKey, WALEdit logEdit) throws IOException {
            ArrayList<Cell> cells = logEdit.getCells();
            for (Cell cell : cells) {
              byte[] fam = CellUtil.cloneFamily(cell);
              if ("f2".equals(Bytes.toString(fam))) {
                NavigableMap<byte[], Integer> scopes = logKey.getScopes();
                if (scopes == null) {
                  logKey.setScopes(new TreeMap<byte[], Integer>(Bytes.BYTES_COMPARATOR));
                }
                logKey.getScopes().put(fam, HConstants.REPLICATION_SCOPE_LOCAL);
              }
            }
            return false;
          }
        }
      

      This logic can not work as org.apache.hadoop.hbase.replication.regionserver.Replication.scopeWALEdits() recreates and populates scopes.
      SOLUTION:
      In Replication.scopeWALEdits(), create scopes map only if WALKey does not have it.

      NavigableMap<byte[], Integer> scopes = logKey.getScopes();
          if (scopes == null) {
            scopes = new TreeMap<byte[], Integer>(Bytes.BYTES_COMPARATOR);
          }
      

        Attachments

        1. HBASE-19423-branch-1.3-001.patch
          8 kB
          Mohammad Arshad
        2. HBASE-19423-branch-1.4-001.patch
          9 kB
          Mohammad Arshad
        3. HBASE-19423-master-001-test.patch
          7 kB
          Mohammad Arshad

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              arshad.mohammad Mohammad Arshad
            • Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: