[HBASE-26630] TableSnapshotInputFormat terminates in the middle

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Duplicate
• Affects Version/s: 2.2.3
• Assignee: Hernan Gelaf-Romer
• Reporter: hyungseok.lim
• Created: 28/Dec/21 06:46
• Updated: 23/Jun/23 16:19
• Resolved: 23/Jun/23 16:19

Description

I am using by adding SingleColumnValueFilter in TableSnapshotInputFormat. In fact, there is a lot of data in the snapshot, but it was found that the mapper was completed in the middle.

 

There was a problem in the next method of ClientSideRegionScanner.

 

 public Result next() throws IOException {
    values.clear();
    scanner.nextRaw(values);
    if (values.isEmpty()) {
      //we are done
      return null;
    }

    Result result = Result.create(values);
    if (this.scanMetrics != null) {
      long resultSize = 0;
      for (Cell cell : values) {
        resultSize += PrivateCellUtil.estimatedSerializedSizeOf(cell);
      }
      this.scanMetrics.countOfBytesInResults.addAndGet(resultSize);
      this.scanMetrics.countOfRowsScanned.incrementAndGet();
    }

    return result;
  } 

values is empty, but scanner.nextRaw(values) returned true.

I modified it as follows and it worked normally.

public Result next() throws IOException {
        values.clear();
        boolean moreValues;
        do {
            moreValues = scanner.nextRaw(values);
        } while (values.isEmpty() && moreValues);

        if (!moreValues) {
            return null;
        }

        Result result = Result.create(values);
        if (this.scanMetrics != null) {
            long resultSize = 0;
            for (Cell cell : values) {
                resultSize += PrivateCellUtil.estimatedSerializedSizeOf(cell);
            }
            this.scanMetrics.countOfBytesInResults.addAndGet(resultSize);
            this.scanMetrics.countOfRowsScanned.incrementAndGet();
        }

        return result;
    } 

Please check this.

 

Issue Links

• duplicates: HBASE-27950 ClientSideRegionScanner does not adhere to RegionScanner.nextRaw contract
• is fixed by: HBASE-26273 TableSnapshotInputFormat/TableSnapshotInputFormatImpl should use ReadType.STREAM for scanning HFiles
• links to: GitHub Pull Request #4215
• links to: GitHub Pull Request #5304