Details

    • Hadoop Flags:
      Reviewed

      Description

      If a storefilescanner fails to seek, it silently swallows the IOE and returns false as if it were the end of the scanner.

      This means that we can silently lose data if an IOE occurs for one of the store files during a compaction.

      1. hbase-2519.txt
        30 kB
        Todd Lipcon
      2. hbase-2519.txt
        27 kB
        Todd Lipcon
      3. hbase-2519-prelim.txt
        16 kB
        Todd Lipcon

        Issue Links

          Activity

          Todd Lipcon created issue -
          Todd Lipcon made changes -
          Field Original Value New Value
          Link This issue is related to HBASE-2231 [ HBASE-2231 ]
          Todd Lipcon made changes -
          Assignee Todd Lipcon [ tlipcon ]
          Hide
          Todd Lipcon added a comment -

          In fact this needs to bubble all the way through to client - right now we also silently return no rows in a scanner in ScannerCallable on the client side if we get an IOE that isn't a DoNotRetryIOException or NSRE.

          Show
          Todd Lipcon added a comment - In fact this needs to bubble all the way through to client - right now we also silently return no rows in a scanner in ScannerCallable on the client side if we get an IOE that isn't a DoNotRetryIOException or NSRE.
          Hide
          stack added a comment -

          In above when you say 'client', you mean the thread orchestrating the compaction or do you mean client using HTable API?

          Show
          stack added a comment - In above when you say 'client', you mean the thread orchestrating the compaction or do you mean client using HTable API?
          Hide
          Todd Lipcon added a comment -

          All the way to the client - see attached patch for example (will write a targeted unit test before commit)

          Show
          Todd Lipcon added a comment - All the way to the client - see attached patch for example (will write a targeted unit test before commit)
          Todd Lipcon made changes -
          Attachment hbase-2519-prelim.txt [ 12443975 ]
          Hide
          stack added a comment -

          Bulk move of 0.20.5 issues into 0.21.0 after vote that we merge branch into TRUNK up on list.

          Show
          stack added a comment - Bulk move of 0.20.5 issues into 0.21.0 after vote that we merge branch into TRUNK up on list.
          stack made changes -
          Fix Version/s 0.21.0 [ 12313607 ]
          Fix Version/s 0.20.5 [ 12314800 ]
          Labels moved_from_0_20_5
          Hide
          Todd Lipcon added a comment -

          Patch and test case attached.

          Show
          Todd Lipcon added a comment - Patch and test case attached.
          Todd Lipcon made changes -
          Attachment hbase-2519.txt [ 12444993 ]
          Todd Lipcon made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hide
          Todd Lipcon added a comment -

          Anyone got time to look at this patch?

          Show
          Todd Lipcon added a comment - Anyone got time to look at this patch?
          Hide
          ryan rawson added a comment -

          it needs to be rebased again unfortunately.

          Show
          ryan rawson added a comment - it needs to be rebased again unfortunately.
          Hide
          Todd Lipcon added a comment -

          Rebased

          Show
          Todd Lipcon added a comment - Rebased
          Todd Lipcon made changes -
          Attachment hbase-2519.txt [ 12445215 ]
          Hide
          Todd Lipcon added a comment -
          Show
          Todd Lipcon added a comment - Review here: http://review.hbase.org/r/75/
          Hide
          Andrew Purtell added a comment -

          +1
          Like the new fault injection tests.

          Show
          Andrew Purtell added a comment - +1 Like the new fault injection tests.
          Hide
          stack added a comment -

          I had two minor nits in review. I can fix them on commit. That ok?

          Show
          stack added a comment - I had two minor nits in review. I can fix them on commit. That ok?
          Hide
          stack added a comment -

          Committed. Thanks for patch Todd. Thanks for review over on review.hbase.org Ryan.

          Show
          stack added a comment - Committed. Thanks for patch Todd. Thanks for review over on review.hbase.org Ryan.
          stack made changes -
          Hadoop Flags [Reviewed]
          Resolution Fixed [ 1 ]
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Hide
          Todd Lipcon added a comment -

          Stack: looks like you missed the test case when you committed this - TestFSErrorsExposed.java isn't in trunk.

          Show
          Todd Lipcon added a comment - Stack: looks like you missed the test case when you committed this - TestFSErrorsExposed.java isn't in trunk.
          Hide
          stack added a comment -

          duh (fixed)

          Show
          stack added a comment - duh (fixed)
          Hide
          Lars Francke added a comment -

          This issue was closed as part of a bulk closing operation on 2015-11-20. All issues that have been resolved and where all fixVersions have been released have been closed (following discussions on the mailing list).

          Show
          Lars Francke added a comment - This issue was closed as part of a bulk closing operation on 2015-11-20. All issues that have been resolved and where all fixVersions have been released have been closed (following discussions on the mailing list).
          Lars Francke made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          Open Open Patch Available Patch Available
          13d 17h 9m 1 Todd Lipcon 19/May/10 22:35
          Patch Available Patch Available Resolved Resolved
          3d 23h 46m 1 stack 23/May/10 22:22
          Resolved Resolved Closed Closed
          2006d 14h 19m 1 Lars Francke 20/Nov/15 12:41

            People

            • Assignee:
              Todd Lipcon
              Reporter:
              Todd Lipcon
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development