Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-13651

Handle StoreFileScanner FileNotFoundException

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 0.94.27, 0.98.10.1
    • Fix Version/s: 0.94.28, 0.98.13, 1.2.0, 2.0.0
    • Component/s: None
    • Labels:
      None

      Description

      Example:

      • Machine-1 is serving Region-X and start compaction
      • Machine-1 goes in GC pause
      • Region-X gets reassigned to Machine-2
      • Machine-1 exit from the GC pause
      • Machine-1 (re)moves the compacted files
      • Machine-1 get the lease expired and shutdown

      Machine-2 has now tons of FileNotFoundException on scan. If we reassign the region everything is ok, because we pickup the files compacted by Machine-1.
      This problem doesn't happen in the new code 1.0+ (i think but I haven't checked, it may be 1.1) where we write on the WAL the compaction event before (re)moving the files.

      A workaround is handling FileNotFoundException and refresh the store files, or shutdown the region and reassign. the first one is easy in 1.0+ the second one requires more work because at the moment we don't have the code to notify the master that the RS is closing the region, alternatively we can shutdown the entire RS (it is not a good solution but the case is rare enough)

        Attachments

        1. HBASE-13651-draft.patch
          21 kB
          Matteo Bertozzi
        2. HBASE-13651-0.94-draft.patch
          15 kB
          Matteo Bertozzi
        3. HBASE-13651-v0-0.94.patch
          15 kB
          Matteo Bertozzi
        4. HBASE-13651-v0-0.98.patch
          15 kB
          Matteo Bertozzi
        5. HBASE-13651-v0-branch-1.patch
          20 kB
          Matteo Bertozzi
        6. HBASE-13651-v0.patch
          20 kB
          Matteo Bertozzi

          Issue Links

            Activity

              People

              • Assignee:
                mbertozzi Matteo Bertozzi
                Reporter:
                mbertozzi Matteo Bertozzi
              • Votes:
                0 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: