[HADOOP-13489] DistCp may incorrectly return success status when the underlying Job failed

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Won't Fix
• Labels: distcp
• Assignee: Ted Yu
• Reporter: Ted Yu
• Created: 11/Aug/16 23:32
• Updated: 18/Jan/17 18:17
• Resolved: 18/Jan/17 18:17

Description

I was troubleshooting HBASE-14450 where at the end of BackupdistCp#execute(), distcp job was marked unsuccessful (BackupdistCp is a wrapper of DistCp).
Yet in IncrementalTableBackupProcedure#incrementalCopy(), the return value from copyService.copy() was 0.

Here is related code from DistCp:

    try {
      execute();
    } catch (InvalidInputException e) {
      LOG.error("Invalid input: ", e);
      return DistCpConstants.INVALID_ARGUMENT;
    } catch (DuplicateFileException e) {
      LOG.error("Duplicate files in input path: ", e);
      return DistCpConstants.DUPLICATE_INPUT;
    } catch (AclsNotSupportedException e) {
      LOG.error("ACLs not supported on at least one file system: ", e);
      return DistCpConstants.ACLS_NOT_SUPPORTED;
    } catch (XAttrsNotSupportedException e) {
      LOG.error("XAttrs not supported on at least one file system: ", e);
      return DistCpConstants.XATTRS_NOT_SUPPORTED;
    } catch (Exception e) {
      LOG.error("Exception encountered ", e);
      return DistCpConstants.UNKNOWN_ERROR;
    }
    return DistCpConstants.SUCCESS;

We don't check whether the Job returned by execute() was successful - we rely on all failure cases going through the last catch clause but there may be special case.
Even if the Job fails, DistCpConstants.SUCCESS is returned.

Issue Links

• is related to: HBASE-14450 HBase Backup/Restore Phase 3: Multiwal support