Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-16752

Upgrading from 1.2 to 1.3 can lead to replication failures due to difference in RPC size limit

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.3.0, 2.0.0
    • Fix Version/s: 1.3.0, 2.0.0
    • Component/s: Replication, rpc
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      In HBase 1.2, we don't limit size of a single RPC but in 1.3 we limit it by default to 256 MB. This means that during upgrade scenarios (or when source is 1.2 peer is already on 1.3), it's possible to encounter a situation where we try to send an rpc with size greater than 256 MB because we never unroll a WALEdit while sending replication traffic.

      RpcServer throws the underlying exception locally, but closes the connection with returning the underlying error to the client, and client only sees a "Broken pipe" error.
      I am not sure what is the proper fix here (or if one is needed) to make sure this does not happen, but we should return the underlying exception to the RpcClient, because without it, it can be difficult to diagnose the problem, especially for someone new to HBase.

        Attachments

        1. HBASE-16752.addendum.patch
          3 kB
          Ashu Pachauri
        2. HBASE-16752.V1.patch
          9 kB
          Ashu Pachauri
        3. HBASE-16752.V2.patch
          9 kB
          Ashu Pachauri

          Issue Links

            Activity

              People

              • Assignee:
                ashu210890 Ashu Pachauri
                Reporter:
                ashu210890 Ashu Pachauri
              • Votes:
                0 Vote for this issue
                Watchers:
                11 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: