Details

    • Type: Sub-task
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.94.0
    • Component/s: None
    • Labels:
      None

      Description

      After incorporating v11 of the 2856 fix, we discovered that we are still having some ACID violations.

      This time, however, the problem is not about including "newer" updates; but, about missing older updates
      that should be including.

      Here is what seems to be happening.

      There is a race condition in the StoreScanner.getScanners()

      private List<KeyValueScanner> getScanners(Scan scan,
      final NavigableSet<byte[]> columns) throws IOException {
      // First the store file scanners
      List<StoreFileScanner> sfScanners = StoreFileScanner
      .getScannersForStoreFiles(store.getStorefiles(), cacheBlocks,
      isGet, false);
      List<KeyValueScanner> scanners =
      new ArrayList<KeyValueScanner>(sfScanners.size()+1);

      // include only those scan files which pass all filters
      for (StoreFileScanner sfs : sfScanners) {
      if (sfs.shouldSeek(scan, columns))

      { scanners.add(sfs); }

      }

      // Then the memstore scanners
      if (this.store.memstore.shouldSeek(scan))

      { scanners.addAll(this.store.memstore.getScanners()); }

      return scanners;
      }

      If for example there is a call to Store.updateStorefiles() that happens between
      the store.getStorefiles() and this.store.memstore.getScanners(); then
      it is possible that there was a new HFile created, that is not seen by the
      StoreScanner, and the data is not present in the Memstore.snapshot either.

        Attachments

        1. 4485-v6.diff
          11 kB
          Amitanand Aiyer
        2. 4485-v5.diff
          11 kB
          Amitanand Aiyer
        3. 4485-v4.diff
          9 kB
          Amitanand Aiyer
        4. 4485-v3.diff
          6 kB
          Amitanand Aiyer
        5. 4485-v2.diff
          6 kB
          Amitanand Aiyer
        6. repro_bug-4485.diff
          1.0 kB
          Amitanand Aiyer
        7. 4485-v1.diff
          0.9 kB
          Amitanand Aiyer

          Issue Links

            Activity

              People

              • Assignee:
                amitanand Amitanand Aiyer
                Reporter:
                amitanand Amitanand Aiyer
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: