HBase
  1. HBase
  2. HBASE-2856 TestAcidGuarantee broken on trunk
  3. HBASE-4345

Ensure that Scanners that read from the storefiles respect MVCC

    Details

    • Type: Sub-task Sub-task
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.89.20100924, 0.94.0
    • Component/s: None
    • Labels:
      None

      Description

      Currently, the key-values written to the disk do not include the MVCC (RWCC) version information. Once we add that
      information, and make it persistent to disk; let us make the scanners respect the MVCC mechanism by ignoring
      "newer" writes.

      1. 4345-v2.txt
        2 kB
        Amitanand Aiyer
      2. patch-3
        8 kB
        Amitanand Aiyer

        Activity

        Hide
        Amitanand Aiyer added a comment -

        To enforce the read-write consistency mechanism, we need to ignore all key-values that have a memstoreTS greater than the read point for the get/scan operation.

        Filters provide an excellent mechanism to ignore values based on any desired condition. We implement the RWCC mechanism for the key-values read from the disk by including a filter for the scan object that ignores "newer" reads.

        Show
        Amitanand Aiyer added a comment - To enforce the read-write consistency mechanism, we need to ignore all key-values that have a memstoreTS greater than the read point for the get/scan operation. Filters provide an excellent mechanism to ignore values based on any desired condition. We implement the RWCC mechanism for the key-values read from the disk by including a filter for the scan object that ignores "newer" reads.
        Hide
        stack added a comment -

        Interesting take, enforcing we only read at the read point or before by doing the functionality as a filter that is always used. Intellectually I'd say that this should be core and not be done via a filter but if filter gets the job done, lets go for it. Nice.

        Show
        stack added a comment - Interesting take, enforcing we only read at the read point or before by doing the functionality as a filter that is always used. Intellectually I'd say that this should be core and not be done via a filter but if filter gets the job done, lets go for it. Nice.
        Hide
        Ted Yu added a comment -

        Minor comment:

        +    @Override
        +    public boolean hasFilterRow() {
        +      if (existingFilter == null)
        +        return false;
        +      else
        +        return existingFilter.hasFilterRow();
        +    }
        

        else is not needed above.

        What's the plan for all the sub-tasks of HBASE-2856 ?

        Show
        Ted Yu added a comment - Minor comment: + @Override + public boolean hasFilterRow() { + if (existingFilter == null ) + return false ; + else + return existingFilter.hasFilterRow(); + } else is not needed above. What's the plan for all the sub-tasks of HBASE-2856 ?
        Hide
        stack added a comment -

        @Amit You have any revisions on this patch since you wrote it? Should we commit?

        Show
        stack added a comment - @Amit You have any revisions on this patch since you wrote it? Should we commit?
        Hide
        Amitanand Aiyer added a comment -

        @stack: I'm going to redo this patch. I gave walk through of this code last week, internally, and one of the
        comments I got was that doing the filter approach is not going to ignore delete-markers that occur after
        the read point. i.e. they will take effect, even though they should not.

        I've been in austin for a recruiting trip, so don't have the patch ready. Will try to get it and update asap.

        Show
        Amitanand Aiyer added a comment - @stack: I'm going to redo this patch. I gave walk through of this code last week, internally, and one of the comments I got was that doing the filter approach is not going to ignore delete-markers that occur after the read point. i.e. they will take effect, even though they should not. I've been in austin for a recruiting trip, so don't have the patch ready. Will try to get it and update asap.
        Hide
        Amitanand Aiyer added a comment -

        I have modified the ScanQueryMatcher to take care of ignoring "newer" KV's. If done by the
        Scan query matcher, it is also going to disregard the deletes so it will help us with other
        issues.

        4344-v8 includes this change.

        Show
        Amitanand Aiyer added a comment - I have modified the ScanQueryMatcher to take care of ignoring "newer" KV's. If done by the Scan query matcher, it is also going to disregard the deletes so it will help us with other issues. 4344-v8 includes this change.
        Hide
        Amitanand Aiyer added a comment -

        fix included in 2856-v6.txt

        Show
        Amitanand Aiyer added a comment - fix included in 2856-v6.txt

          People

          • Assignee:
            Amitanand Aiyer
            Reporter:
            Amitanand Aiyer
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development