Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-12642

LoadIncrementalHFiles does not throw exception after hitting hbase.bulkload.retries.number setting

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 0.98.1
    • Fix Version/s: 1.0.0, 0.98.9
    • Component/s: mapreduce
    • Labels:
      None
    • Environment:

      RedHat 6.6
      Java 1.7.0_65
      HBase 0.98.1-cdh5.1.0
      Hadoop 2.3.0-mr1-cdh5.1.0

    • Hadoop Flags:
      Reviewed

      Description

      Related to HBASE-8367 there is no programmatic way to determine if the hbase.bulkload.retries.number value has been hit. Callers of doBulkLoad have no way to determine other than log examination whether the bulk load was actually successful.

      Here is the relevant code sections:

      LoadIncrementalHFiles.java
      public void doBulkLoad ...
      {
          try {
          ...
              if (maxRetries != 0 && count >= maxRetries) {
                  LOG.error("Retry attempted " + count + " times without completing, bailing out");
                  return;
              }
          ...
          } finally {
          ...
            pool.shutdown();
            if (queue != null && !queue.isEmpty()) {
              StringBuilder err = new StringBuilder();
              err.append("-------------------------------------------------\n");
              err.append("Bulk load aborted with some files not yet loaded:\n");
              err.append("-------------------------------------------------\n");
              for (LoadQueueItem q : queue) {
                err.append("  ").append(q.hfilePath).append('\n');
              }
              LOG.error(err);
            }
          }
      
          if (queue != null && !queue.isEmpty()) {
              throw new RuntimeException("Bulk load aborted with some files not yet loaded."
                + "Please check log for more details.");
      }
      

      One possible solution is to move the throw new RuntimeException clause at the end into the finally block. The exception is currently never thrown for max retries due to the return in the initial if statement.

        Attachments

          Activity

            People

            • Assignee:
              yuzhihong@gmail.com Ted Yu
              Reporter:
              eingle Erik Ingle
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: