HBase
  1. HBase
  2. HBASE-11114

Backport HBASE-10926 (Use global procedure to flush table memstore cache) to 0.98

    Details

    • Type: Task Task
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Later
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      Backport HBASE-10926 to 0.98.

      Description from original issue: Currently, user can trigger table flush through hbase shell or HBaseAdmin API. To flush the table cache, each region server hosting the regions is contacted and flushed sequentially, which is less efficient. In HBase snapshot global procedure is used to coordinate and flush the regions in a distributed way. Let's provide a distributed table flush for general use.

        Issue Links

          Activity

          Hide
          Andrew Purtell added a comment -

          I'm going to resolve this as Later as in retrospect there is no priority need for this feature. Let's reopen when being actively worked on.

          Show
          Andrew Purtell added a comment - I'm going to resolve this as Later as in retrospect there is no priority need for this feature. Let's reopen when being actively worked on.
          Hide
          Jerry He added a comment -

          Old HBaseAdmin clients (say 0.98.0) who use the old style global flush should be able to issue the commands to newer servers (0.98.3 let's say).

          This should work fine if we backport directly. Old client will still go to region servers to flush without involving the global procedure.

          New HBaseAdmin clients (say 0.98.3) could try to do the new style flush to old servers (0.98.00 but should fall back to the old style if it is rejected with some sort of not supported error.)

          This will break.

          Show
          Jerry He added a comment - Old HBaseAdmin clients (say 0.98.0) who use the old style global flush should be able to issue the commands to newer servers (0.98.3 let's say). This should work fine if we backport directly. Old client will still go to region servers to flush without involving the global procedure. New HBaseAdmin clients (say 0.98.3) could try to do the new style flush to old servers (0.98.00 but should fall back to the old style if it is rejected with some sort of not supported error.) This will break.
          Hide
          Andrew Purtell added a comment -

          Push to 0.98.4. The compatibility issues will need to be well tested.

          Show
          Andrew Purtell added a comment - Push to 0.98.4. The compatibility issues will need to be well tested.
          Hide
          Andrew Purtell added a comment -

          Sounds good Jon.

          Show
          Andrew Purtell added a comment - Sounds good Jon.
          Hide
          Jonathan Hsieh added a comment -

          The frame work should be compat.

          I think we can go case by case with the commands that use it. Here are some suggested rules:

          1. old clients should be able to issue the command to new servers and new clients should be able to issue to the command to old servers and still succeed. Also, they should succeed in mixed old/new RS situations (e.g. while we are doing a rolling upgrade).

          2. I'm ok with having a flag for old behavior and all new clients in 0.98.x default to old behavior. Switching to the new requires full shutdown/restart unless #1 is met as well.

          Let me give a more concrete example of a compat requirement for a global flush method:

          • Old HBaseAdmin clients (say 0.98.0) who use the old style global flush should be able to issue the commands to newer servers (0.98.3 let's say).
          • New HBaseAdmin clients (say 0.98.3) could try to do the new style flush to old servers (0.98.00 but should fall back to the old style if it is rejected with some sort of not supported error.)
          Show
          Jonathan Hsieh added a comment - The frame work should be compat. I think we can go case by case with the commands that use it. Here are some suggested rules: 1. old clients should be able to issue the command to new servers and new clients should be able to issue to the command to old servers and still succeed. Also, they should succeed in mixed old/new RS situations (e.g. while we are doing a rolling upgrade). 2. I'm ok with having a flag for old behavior and all new clients in 0.98.x default to old behavior. Switching to the new requires full shutdown/restart unless #1 is met as well. Let me give a more concrete example of a compat requirement for a global flush method: Old HBaseAdmin clients (say 0.98.0) who use the old style global flush should be able to issue the commands to newer servers (0.98.3 let's say). New HBaseAdmin clients (say 0.98.3) could try to do the new style flush to old servers (0.98.00 but should fall back to the old style if it is rejected with some sort of not supported error.)
          Hide
          Andrew Purtell added a comment -

          The coprocessor API changes in the committed patch for HBASE-11114 add methods only, so would be ok to port back. I think the procedure framework in 0.98 can support the changes also.

          I think the one question is if the flush behavior change is ok to make from one minor release to another. In my opinion we should take a liberal attitude to changes that improve efficiency without changing client facing API, and when security is active flushing is a restricted activity already. A release note should be sufficient.

          Show
          Andrew Purtell added a comment - The coprocessor API changes in the committed patch for HBASE-11114 add methods only, so would be ok to port back. I think the procedure framework in 0.98 can support the changes also. I think the one question is if the flush behavior change is ok to make from one minor release to another. In my opinion we should take a liberal attitude to changes that improve efficiency without changing client facing API, and when security is active flushing is a restricted activity already. A release note should be sufficient.

            People

            • Assignee:
              Unassigned
              Reporter:
              Andrew Purtell
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development