XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 0.94.0
    • None
    • None

    Description

      After incorporating v11 of the 2856 fix, we discovered that we are still having some ACID violations.

      This time, however, the problem is not about including "newer" updates; but, about missing older updates
      that should be including.

      Here is what seems to be happening.

      There is a race condition in the StoreScanner.getScanners()

      private List<KeyValueScanner> getScanners(Scan scan,
      final NavigableSet<byte[]> columns) throws IOException {
      // First the store file scanners
      List<StoreFileScanner> sfScanners = StoreFileScanner
      .getScannersForStoreFiles(store.getStorefiles(), cacheBlocks,
      isGet, false);
      List<KeyValueScanner> scanners =
      new ArrayList<KeyValueScanner>(sfScanners.size()+1);

      // include only those scan files which pass all filters
      for (StoreFileScanner sfs : sfScanners) {
      if (sfs.shouldSeek(scan, columns))

      { scanners.add(sfs); }

      }

      // Then the memstore scanners
      if (this.store.memstore.shouldSeek(scan))

      { scanners.addAll(this.store.memstore.getScanners()); }

      return scanners;
      }

      If for example there is a call to Store.updateStorefiles() that happens between
      the store.getStorefiles() and this.store.memstore.getScanners(); then
      it is possible that there was a new HFile created, that is not seen by the
      StoreScanner, and the data is not present in the Memstore.snapshot either.

      Attachments

        1. 4485-v1.diff
          0.9 kB
          Amitanand Aiyer
        2. 4485-v2.diff
          6 kB
          Amitanand Aiyer
        3. 4485-v3.diff
          6 kB
          Amitanand Aiyer
        4. 4485-v4.diff
          9 kB
          Amitanand Aiyer
        5. 4485-v5.diff
          11 kB
          Amitanand Aiyer
        6. 4485-v6.diff
          11 kB
          Amitanand Aiyer
        7. repro_bug-4485.diff
          1.0 kB
          Amitanand Aiyer

        Issue Links

          Activity

            People

              amitanand Amitanand Aiyer
              amitanand Amitanand Aiyer
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: