[HBASE-13254] EnableTableHandler#prepare would not throw TableNotFoundException during recovery

• Type: Bug
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 2.0.0
• Fix Version/s: 2.0.0
• Hadoop Flags: Reviewed
• Assignee: Stephen Yuan Jiang
• Reporter: Stephen Yuan Jiang
• Created: 16/Mar/15 21:23
• Updated: 23/Jun/22 20:41
• Resolved: 17/Mar/15 00:50

Description

During recovery, when EnableTableHandler#prepare() is called, If the table does not exist, it marks the table as deleted and does NOT throw TableNotFoundException. The result is that the table lock is released and the caller has no knowledge that the table not exist or already deleted, it would continue the next step.

  public EnableTableHandler prepare()
      throws TableNotFoundException, TableNotDisabledException, IOException {
   ...
    try {
      // Check if table exists
      if (!MetaTableAccessor.tableExists(this.server.getConnection(), tableName)) {
        // retainAssignment is true only during recovery.  In normal case it is false
        if (!this.skipTableStateCheck) {
          throw new TableNotFoundException(tableName);
        }
         this.assignmentManager.getTableStateManager().setDeletedTable(tableName);
      }
   ...
  }

However,look at the recovery code that calls the EnableTableHandler#prepare function, AssignmentManager#recoverTableInEnablingState() expects TableNotFoundException so that it can skip the table.

  private void recoverTableInEnablingState()
          throws KeeperException, IOException {
    Set<TableName> enablingTables = tableStateManager.
            getTablesInStates(TableState.State.ENABLING);
    if (enablingTables.size() != 0) {
      for (TableName tableName : enablingTables) {
        // Recover by calling EnableTableHandler
        LOG.info("The table " + tableName
            + " is in ENABLING state.  Hence recovering by moving the table"
            + " to ENABLED state.");
        // enableTable in sync way during master startup,
        // no need to invoke coprocessor
        EnableTableHandler eth = new EnableTableHandler(this.server, tableName,
          this, tableLockManager, true);
        try {
          eth.prepare();
        } catch (TableNotFoundException e) {
          LOG.warn("Table " + tableName + " not found in hbase:meta to recover.");
          continue;
        }
        eth.process();
      }
    }
  }

The proposed fix is always throw TableNotFoundException in EnableTableHandler#prepare if the table does not exist.

note: this bug only applies to master, a regression from HBASE-7767. Branch-1 has the correct logic, after setting table state, it would throw the exception.