Details

    • Type: Sub-task Sub-task
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.94.0
    • Component/s: None
    • Labels:
      None

      Description

      After incorporating v11 of the 2856 fix, we discovered that we are still having some ACID violations.

      This time, however, the problem is not about including "newer" updates; but, about missing older updates
      that should be including.

      Here is what seems to be happening.

      There is a race condition in the StoreScanner.getScanners()

      private List<KeyValueScanner> getScanners(Scan scan,
      final NavigableSet<byte[]> columns) throws IOException {
      // First the store file scanners
      List<StoreFileScanner> sfScanners = StoreFileScanner
      .getScannersForStoreFiles(store.getStorefiles(), cacheBlocks,
      isGet, false);
      List<KeyValueScanner> scanners =
      new ArrayList<KeyValueScanner>(sfScanners.size()+1);

      // include only those scan files which pass all filters
      for (StoreFileScanner sfs : sfScanners) {
      if (sfs.shouldSeek(scan, columns))

      { scanners.add(sfs); }

      }

      // Then the memstore scanners
      if (this.store.memstore.shouldSeek(scan))

      { scanners.addAll(this.store.memstore.getScanners()); }

      return scanners;
      }

      If for example there is a call to Store.updateStorefiles() that happens between
      the store.getStorefiles() and this.store.memstore.getScanners(); then
      it is possible that there was a new HFile created, that is not seen by the
      StoreScanner, and the data is not present in the Memstore.snapshot either.

      1. 4485-v6.diff
        11 kB
        Amitanand Aiyer
      2. 4485-v5.diff
        11 kB
        Amitanand Aiyer
      3. 4485-v4.diff
        9 kB
        Amitanand Aiyer
      4. 4485-v3.diff
        6 kB
        Amitanand Aiyer
      5. 4485-v2.diff
        6 kB
        Amitanand Aiyer
      6. repro_bug-4485.diff
        1.0 kB
        Amitanand Aiyer
      7. 4485-v1.diff
        0.9 kB
        Amitanand Aiyer

        Issue Links

          Activity

          No work has yet been logged on this issue.

            People

            • Assignee:
              Amitanand Aiyer
              Reporter:
              Amitanand Aiyer
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development