Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-1738

Scanner doesnt reset when a snapshot is created, could miss new updates into the 'kvset' (active part)

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.20.0
    • Fix Version/s: 0.20.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      when a Scanner is created, it creates 2 MemStoreScanners on the kvset and the snapshot (internal names of Memstore)... if the snapshot is originally empty, it only creates the 1, for kvset. When the snapshot is created, the outstanding Scanners now have a pointer to the tree that is now the snapshot, but no pointer to the kvset.

      When the flush completes, the scanner will reset the memstore scanners and 'see' the new values again.

      If there is a large delay between snapshot and finalization of the flush, there can be a large period of time a scanner doesnt see 'new' values that are being inserted. the canonical 'bad' case where this can do things is the META scanner, and we end up with double assignment.

      The snapshot is really lightweight, it only takes out a small lock in memstore, so im not sure there is an easy mechanism to hook to without building out a bit more code or restructuring the memstore scanner.

        Attachments

        1. 1738.patch
          8 kB
          stack
        2. 1738-v2.patch
          10 kB
          stack
        3. 1738-v3.patch
          11 kB
          stack
        4. 1738-v3-withheapsizefix.patch
          14 kB
          Jonathan Gray

          Activity

            People

            • Assignee:
              stack stack
              Reporter:
              ryanobjc ryan rawson
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: