Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-20604

ProtobufLogReader#readNext can incorrectly loop to the same position in the stream until the the WAL is rolled

    XMLWordPrintableJSON

    Details

      Description

      Every time we call ProtobufLogReader#readNext we consume the input stream associated to the FSDataInputStream from the WAL that we are reading. Under certain conditions, e.g. when using the encryption at rest (CryptoInputStream) the stream can return partial data which can cause a premature EOF that cause inputStream.getPos() to return to the same origina position causing ProtobufLogReader#readNext to re-try over the reads until the WAL is rolled.

      The side effect of this issue is that ReplicationSource can get stuck until the WAL is rolled and causing replication delays up to an hour in some cases.

        Attachments

        1. HBASE-20604.patch
          3 kB
          Esteban Gutierrez
        2. HBASE-20604.002.patch
          3 kB
          Esteban Gutierrez
        3. HBASE-20604.003.patch
          4 kB
          Esteban Gutierrez
        4. HBASE-20604.004.patch
          4 kB
          Esteban Gutierrez
        5. HBASE-20604.005.patch
          4 kB
          Esteban Gutierrez

          Activity

            People

            • Assignee:
              esteban Esteban Gutierrez
              Reporter:
              esteban Esteban Gutierrez
            • Votes:
              0 Vote for this issue
              Watchers:
              17 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: