Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-17292

Add observer notification before bulk loaded hfile is moved to region directory

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 1.4.0, 2.0.0
    • None
    • None
    • Reviewed

    Description

      Currently the postBulkLoadHFile() hook notifies the locations of bulk loaded hfiles.
      However, if bulk load fails after hfile is moved to region directory but before postBulkLoadHFile() hook is called, there is no way for pluggable components (replication - see HBASE-17290, backup / restore) to know which hfile(s) have been moved to region directory.

      Even if postBulkLoadHFile() is called in finally block, the write (to backup table or zookeeper) issued from postBulkLoadHFile() may fail, ending up with same situation.

      This issue adds a preCommitStoreFile() hook which notifies path of to be committed hfile before bulk loaded hfile is moved to region directory.

      With preCommitStoreFile() hook, write (to backup table or zookeeper) can be issued before the movement of hfile.
      If write fails, IOException would make bulk load fail, not leaving hfile in region directory.

      Attachments

        1. 17292.v3.txt
          20 kB
          Ted Yu
        2. 17292.v2.txt
          14 kB
          Ted Yu
        3. 17292.v1.txt
          13 kB
          Ted Yu
        4. 17292.branch-1.v1.txt
          17 kB
          Ted Yu
        5. 17292.addendum
          6 kB
          Ted Yu

        Activity

          People

            yuzhihong@gmail.com Ted Yu
            yuzhihong@gmail.com Ted Yu
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: