Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-20604

ProtobufLogReader#readNext can incorrectly loop to the same position in the stream until the the WAL is rolled

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    Description

      Every time we call ProtobufLogReader#readNext we consume the input stream associated to the FSDataInputStream from the WAL that we are reading. Under certain conditions, e.g. when using the encryption at rest (CryptoInputStream) the stream can return partial data which can cause a premature EOF that cause inputStream.getPos() to return to the same origina position causing ProtobufLogReader#readNext to re-try over the reads until the WAL is rolled.

      The side effect of this issue is that ReplicationSource can get stuck until the WAL is rolled and causing replication delays up to an hour in some cases.

      Attachments

        1. HBASE-20604.005.patch
          4 kB
          Esteban Gutierrez
        2. HBASE-20604.004.patch
          4 kB
          Esteban Gutierrez
        3. HBASE-20604.003.patch
          4 kB
          Esteban Gutierrez
        4. HBASE-20604.002.patch
          3 kB
          Esteban Gutierrez
        5. HBASE-20604.patch
          3 kB
          Esteban Gutierrez

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            esteban Esteban Gutierrez
            esteban Esteban Gutierrez
            Votes:
            0 Vote for this issue
            Watchers:
            17 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment