HBase
  1. HBase
  2. HBASE-2519

StoreFileScanner.seek swallows IOEs

    Details

    • Hadoop Flags:
      Reviewed

      Description

      If a storefilescanner fails to seek, it silently swallows the IOE and returns false as if it were the end of the scanner.

      This means that we can silently lose data if an IOE occurs for one of the store files during a compaction.

      1. hbase-2519.txt
        30 kB
        Todd Lipcon
      2. hbase-2519.txt
        27 kB
        Todd Lipcon
      3. hbase-2519-prelim.txt
        16 kB
        Todd Lipcon

        Issue Links

          Activity

          Hide
          Todd Lipcon added a comment -

          In fact this needs to bubble all the way through to client - right now we also silently return no rows in a scanner in ScannerCallable on the client side if we get an IOE that isn't a DoNotRetryIOException or NSRE.

          Show
          Todd Lipcon added a comment - In fact this needs to bubble all the way through to client - right now we also silently return no rows in a scanner in ScannerCallable on the client side if we get an IOE that isn't a DoNotRetryIOException or NSRE.
          Hide
          stack added a comment -

          In above when you say 'client', you mean the thread orchestrating the compaction or do you mean client using HTable API?

          Show
          stack added a comment - In above when you say 'client', you mean the thread orchestrating the compaction or do you mean client using HTable API?
          Hide
          Todd Lipcon added a comment -

          All the way to the client - see attached patch for example (will write a targeted unit test before commit)

          Show
          Todd Lipcon added a comment - All the way to the client - see attached patch for example (will write a targeted unit test before commit)
          Hide
          stack added a comment -

          Bulk move of 0.20.5 issues into 0.21.0 after vote that we merge branch into TRUNK up on list.

          Show
          stack added a comment - Bulk move of 0.20.5 issues into 0.21.0 after vote that we merge branch into TRUNK up on list.
          Hide
          Todd Lipcon added a comment -

          Patch and test case attached.

          Show
          Todd Lipcon added a comment - Patch and test case attached.
          Hide
          Todd Lipcon added a comment -

          Anyone got time to look at this patch?

          Show
          Todd Lipcon added a comment - Anyone got time to look at this patch?
          Hide
          ryan rawson added a comment -

          it needs to be rebased again unfortunately.

          Show
          ryan rawson added a comment - it needs to be rebased again unfortunately.
          Hide
          Todd Lipcon added a comment -

          Rebased

          Show
          Todd Lipcon added a comment - Rebased
          Hide
          Todd Lipcon added a comment -
          Show
          Todd Lipcon added a comment - Review here: http://review.hbase.org/r/75/
          Hide
          Andrew Purtell added a comment -

          +1
          Like the new fault injection tests.

          Show
          Andrew Purtell added a comment - +1 Like the new fault injection tests.
          Hide
          stack added a comment -

          I had two minor nits in review. I can fix them on commit. That ok?

          Show
          stack added a comment - I had two minor nits in review. I can fix them on commit. That ok?
          Hide
          stack added a comment -

          Committed. Thanks for patch Todd. Thanks for review over on review.hbase.org Ryan.

          Show
          stack added a comment - Committed. Thanks for patch Todd. Thanks for review over on review.hbase.org Ryan.
          Hide
          Todd Lipcon added a comment -

          Stack: looks like you missed the test case when you committed this - TestFSErrorsExposed.java isn't in trunk.

          Show
          Todd Lipcon added a comment - Stack: looks like you missed the test case when you committed this - TestFSErrorsExposed.java isn't in trunk.
          Hide
          stack added a comment -

          duh (fixed)

          Show
          stack added a comment - duh (fixed)

            People

            • Assignee:
              Todd Lipcon
              Reporter:
              Todd Lipcon
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development