[HBASE-25501] Backup not using parameters such as bandwidth, workers, etc while exporting snapshot

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.0.0-alpha-1
• Fix Version/s: 3.0.0-alpha-1
• Component/s: backup&restore
• Hadoop Flags: Reviewed
• Assignee: Mallikarjun
• Reporter: Mallikarjun
• Created: 13/Jan/21 07:00
• Updated: 27/Jan/21 12:54
• Resolved: 26/Jan/21 06:40

Description

 

// code placeholder
$ sudo /etc/init.d/yak master hbase backup create 
Please make sure that backup is enabled on the cluster. To enable backup, in hbase-site.xml, set:
 hbase.backup.enable=true
hbase.master.logcleaner.plugins=YOUR_PLUGINS,org.apache.hadoop.hbase.backup.master.BackupLogCleaner
hbase.procedure.master.classes=YOUR_CLASSES,org.apache.hadoop.hbase.backup.master.LogRollMasterProcedureManager
hbase.procedure.regionserver.classes=YOUR_CLASSES,org.apache.hadoop.hbase.backup.regionserver.LogRollRegionServerProcedureManager
hbase.coprocessor.region.classes=YOUR_CLASSES,org.apache.hadoop.hbase.backup.BackupObserver
and restart the clusterUsage: hbase backup create <type> <backup_path> [options]
  type           "full" to create a full backup image
                 "incremental" to create an incremental backup image
  backup_path     Full path to store the backup imageOptions:
  -b <arg>        Bandwidth per task (MapReduce task) in MB/s
  -d              Enable debug loggings
  -q <arg>        Yarn queue name to run backup create command on
  -s <arg>        Backup set to backup, mutually exclusive with -t (table list)
  -t <arg>        Table name list, comma-separated.
  -w <arg>        Number of parallel MapReduce tasks to execute 

Parameters -b, -q, -w are not being used when creating a export snapshot request 

// code placeholder

for (TableName table : backupInfo.getTables()) {
  // Currently we simply set the sub copy tasks by counting the table snapshot number, we can
  // calculate the real files' size for the percentage in the future.
  // backupCopier.setSubTaskPercntgInWholeTask(1f / numOfSnapshots);
  int res;
  String[] args = new String[4];
  args[0] = "-snapshot";
  args[1] = backupInfo.getSnapshotName(table);
  args[2] = "-copy-to";
  args[3] = backupInfo.getTableBackupDir(table);

  String jobname = "Full-Backup_" + backupInfo.getBackupId() + "_" + table.getNameAsString();
  if (LOG.isDebugEnabled()) {
    LOG.debug("Setting snapshot copy job name to : " + jobname);
  }
  conf.set(JOB_NAME_CONF_KEY, jobname);

  LOG.debug("Copy snapshot " + args[1] + " to " + args[3]);
  res = copyService.copy(backupInfo, backupManager, conf, BackupType.FULL, args);

  // if one snapshot export failed, do not continue for remained snapshots
  if (res != 0) {
    LOG.error("Exporting Snapshot " + args[1] + " failed with return code: " + res + ".");

    throw new IOException("Failed of exporting snapshot " + args[1] + " to " + args[3]
        + " with reason code " + res);
  }

  conf.unset(JOB_NAME_CONF_KEY);
  LOG.info("Snapshot copy " + args[1] + " finished.");
} 

Issue Links

• is part of: HBASE-17362 HBase Backup/Restore Phase 4
• links to: GitHub Pull Request #2886