HBase
  1. HBase
  2. HBASE-8533

HBaseAdmin does not ride over cluster restart

    Details

    • Type: Improvement Improvement
    • Status: Open
    • Priority: Minor Minor
    • Resolution: Unresolved
    • Affects Version/s: 0.98.0, 0.95.0
    • Fix Version/s: 2.0.0
    • Component/s: Client
    • Labels:
      None

      Description

      For Restful servlet (org.apache.hadoop.hbase.rest.Main (0.94), org.apache.hadoop.hbase.rest.RESTServer (trunk)) on Jetty, we need to first explicitly start the service (% ./bin/hbase-daemon.sh start rest -p 8000 ) for application running. Here is a scenario, sometimes, HBase cluster are stopped/started for maintanence, but rest is a seperated standalone process, which binds the HBaseAdmin at construction method.
      HBase stop/start cause this binding lost for existing rest servlet. Rest servlet still exist to trying on old bound HBaseAdmin until a long time duration later with an "Unavailable" caught via an IOException caught in
      such as RootResource.

      Could we pairwise the HBase service with HBase rest service with some start/stop options? since seems no reason to still keep the rest servlet process after HBase stopped? When HBase restarts, original rest service could not resume to bind to the new HBase service via its old HBaseAdmin reference?
      So may we stop the rest when hbase stopped, or even if hbase was killed by acident, restart hbase with rest option could detect the old rest process, kill it and start to bind a new one?

      From this point of view, application rely on rest api in previous scenario could immediately detect it when setting up http connection session instead of wasting a long time to fail back from IOException with "Unavailable" from rest servlet.

      Put current options from the discussion history here from Andrew, Stack and Jean-Daniel,
      1) create an HBaseAdmin on demand in rest servlet instead of keeping singleton instance; (another possible enhancement for HBase client: automatic reconnection of an open HBaseAdmin handle after a cluster bounce?)
      2) pairwise the rest webapp with hbase webui so the rest is always on with HBase serive;
      3) add an option for rest service (such as HBASE_MANAGES_REST) in hbase-env.sh, set HBASE_MANAGES_REST to true, the scripts will start/stop the REST server.

      1. 8533-0.95-v1.patch
        3 kB
        Julian Zhou
      2. 8533-trunk-v1.patch
        3 kB
        Ted Yu
      3. hbase-8533-trunk-v0.patch
        3 kB
        Julian Zhou

        Issue Links

          Activity

          Sean Busbey made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Fix Version/s 1.2.0 [ 12332062 ]
          Nick Dimiduk made changes -
          Fix Version/s 2.0.0 [ 12327188 ]
          Fix Version/s 1.2.0 [ 12332062 ]
          Fix Version/s 1.1.0 [ 12329043 ]
          Enis Soztutar made changes -
          Fix Version/s 1.1.0 [ 12329043 ]
          Fix Version/s 1.0.0 [ 12325852 ]
          Enis Soztutar made changes -
          Fix Version/s 1.0.0 [ 12325852 ]
          Fix Version/s 0.99.2 [ 12328822 ]
          Enis Soztutar made changes -
          Fix Version/s 0.99.2 [ 12328822 ]
          Fix Version/s 0.99.1 [ 12328551 ]
          Enis Soztutar made changes -
          Fix Version/s 0.99.1 [ 12328551 ]
          Fix Version/s 0.99.0 [ 12325675 ]
          stack made changes -
          Fix Version/s 0.99.0 [ 12325675 ]
          Fix Version/s 0.96.1 [ 12324961 ]
          stack made changes -
          Fix Version/s 0.96.1 [ 12324961 ]
          Fix Version/s 0.98.0 [ 12323143 ]
          Fix Version/s 0.95.0 [ 12324094 ]
          Julian Zhou made changes -
          Component/s Client [ 12312131 ]
          Julian Zhou made changes -
          Summary [REST] HBaseAdmin does not ride over cluster restart HBaseAdmin does not ride over cluster restart
          Julian Zhou made changes -
          Component/s REST [ 12312140 ]
          Component/s scripts [ 12312141 ]
          Ted Yu made changes -
          Assignee Julian Zhou [ julian.zhou ]
          Julian Zhou made changes -
          Attachment 8533-0.95-v1.patch [ 12584868 ]
          Julian Zhou made changes -
          Fix Version/s 0.95.0 [ 12324094 ]
          Fix Version/s 0.94.3 [ 12323144 ]
          Julian Zhou made changes -
          Affects Version/s 0.95.0 [ 12324094 ]
          Affects Version/s 0.94.3 [ 12323144 ]
          Ted Yu made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Ted Yu made changes -
          Attachment 8533-trunk-v1.patch [ 12584821 ]
          Julian Zhou made changes -
          Attachment hbase-8533-trunk-v0.patch [ 12583454 ]
          Julian Zhou made changes -
          Link This issue is related to HBASE-6523 [ HBASE-6523 ]
          Julian Zhou made changes -
          Link This issue is related to HBASE-7469 [ HBASE-7469 ]
          Andrew Purtell made changes -
          Summary Pairwise HBase rest service with HBase serivce. [REST] HBaseAdmin does not ride over cluster restart
          Julian Zhou made changes -
          Field Original Value New Value
          Description For Restful servlet (org.apache.hadoop.hbase.rest.Main (0.94),
          org.apache.hadoop.hbase.rest.RESTServer (trunk)) on Jetty, we need to
          first explicitly start the service (% ./bin/hbase-daemon.sh start rest
          -p 8000 ) for application running. Here is a scenario, sometimes, HBase
          cluster are stopped/started for maintanence, but rest is a seperated
          standalone process, which binds the HBaseAdmin at construction method.
          HBase stop/start cause this binding lost for existing rest servlet. Rest
          servlet still exist to trying on old bound HBaseAdmin until a long time
          duration later with an "Unavailable" caught via an IOException caught in
          such as RootResource.

          Could we pairwise the HBase service with HBase rest service with some
          start/stop options? since seems no reason to still keep the rest servlet
          process after HBase stopped? When HBase restarts, original rest service
          could not resume to bind to the new HBase service via its old HBaseAdmin
          reference?
          So may we stop the rest when hbase stopped, or even if hbase was killed
          by acident, restart hbase with rest option could detect the old rest
          process, kill it and start to bind a new one?

          From this point of view, application rely on rest api in previous
          scenario could immediately detect it when setting up http connection
          session instead of wasting a long time to fail back from IOException
          with "Unavailable" from rest servlet.
          For Restful servlet (org.apache.hadoop.hbase.rest.Main (0.94), org.apache.hadoop.hbase.rest.RESTServer (trunk)) on Jetty, we need to first explicitly start the service (% ./bin/hbase-daemon.sh start rest -p 8000 ) for application running. Here is a scenario, sometimes, HBase cluster are stopped/started for maintanence, but rest is a seperated standalone process, which binds the HBaseAdmin at construction method.
          HBase stop/start cause this binding lost for existing rest servlet. Rest servlet still exist to trying on old bound HBaseAdmin until a long time duration later with an "Unavailable" caught via an IOException caught in
          such as RootResource.

          Could we pairwise the HBase service with HBase rest service with some start/stop options? since seems no reason to still keep the rest servlet process after HBase stopped? When HBase restarts, original rest service could not resume to bind to the new HBase service via its old HBaseAdmin reference?
          So may we stop the rest when hbase stopped, or even if hbase was killed by acident, restart hbase with rest option could detect the old rest process, kill it and start to bind a new one?

          From this point of view, application rely on rest api in previous scenario could immediately detect it when setting up http connection session instead of wasting a long time to fail back from IOException with "Unavailable" from rest servlet.

          Put current options from the discussion history here from Andrew, Stack and Jean-Daniel,
          1) create an HBaseAdmin on demand in rest servlet instead of keeping singleton instance; (another possible enhancement for HBase client: automatic reconnection of an open HBaseAdmin handle after a cluster bounce?)
          2) pairwise the rest webapp with hbase webui so the rest is always on with HBase serive;
          3) add an option for rest service (such as HBASE_MANAGES_REST) in hbase-env.sh, set HBASE_MANAGES_REST to true, the scripts will start/stop the REST server.
          Julian Zhou created issue -

            People

            • Assignee:
              Julian Zhou
              Reporter:
              Julian Zhou
            • Votes:
              0 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

              • Created:
                Updated:

                Development