[HBASE-20769] getSplits() has a out of bounds problem in TableSnapshotInputFormatImpl

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.3.0, 1.4.0, 2.0.0
• Fix Version/s: 3.0.0-alpha-1, 2.1.0, 1.3.3, 1.4.6, 2.0.2
• Hadoop Flags: Reviewed
• Assignee: Jingyun Tian
• Reporter: Jingyun Tian
• Created: 21/Jun/18 10:44
• Updated: 04/Mar/19 09:37
• Resolved: 29/Jun/18 22:55

Description

When numSplits > 1, getSplits may create split that has start row smaller than user specified scan's start row or stop row larger than user specified scan's stop row.

        byte[][] sp = sa.split(hri.getStartKey(), hri.getEndKey(), numSplits, true);
        for (int i = 0; i < sp.length - 1; i++) {
          if (PrivateCellUtil.overlappingKeys(scan.getStartRow(), scan.getStopRow(), sp[i],
                  sp[i + 1])) {
            List<String> hosts =
                calculateLocationsForInputSplit(conf, htd, hri, tableDir, localityEnabled);

            Scan boundedScan = new Scan(scan);
            boundedScan.setStartRow(sp[i]);
            boundedScan.setStopRow(sp[i + 1]);

            splits.add(new InputSplit(htd, hri, hosts, boundedScan, restoreDir));
          }
        }

Since we split keys by the range of regions, when sp[i] < scan.getStartRow() or sp[i + 1] > scan.getStopRow(), the created bounded scan may contain range that over user defined scan.

fix should be simple:

boundedScan.setStartRow(
 Bytes.compareTo(scan.getStartRow(), sp[i]) > 0 ? scan.getStartRow() : sp[i]);
 boundedScan.setStopRow(
 Bytes.compareTo(scan.getStopRow(), sp[i + 1]) < 0 ? scan.getStopRow() : sp[i + 1]);

I will also try to add UTs to help discover this problem

Issue Links

• is related to: HBASE-18090 Improve TableSnapshotInputFormat to allow more multiple mappers per region
• links to: Review Board (branch-1)
• links to: Review Board (master)