Uploaded image for project: 'Kudu'
  1. Kudu
  2. KUDU-3023

Enforce consistency between maximum RPC size and maximum size of transaction

Agile BoardAttach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    Description

      It seems that with the absence of constraints for the --rpc_max_message_size and the --tablet_transaction_memory flags, there might a transaction that can get into the WALs of tablet replicas, but no replica can apply the transaction.

      It would be nice to clarify on this and if necessary, establish a group validator to prevent such situations from happening, if possible.

      Few other points to clarify:

      • What if the data in RPC message that client sends to leader replica is compressed? Can we guarantee that corresponding Raft update sent to followers after persisting the data in the leader's WAL is compressed as well?
      • How much cruft is added by converting the incoming data into a WAL entry and then into a Raft update RPC message? Is it possible for a leader replica to accept an update on the tablet since it's under the max RPC size limit, but be unable push corresponding Raft update message to follower replicas because of some extra cruft added when converting the accepted WriteRequestPB into ConsensusRequestPB containing ReplicateMsg with embedded original WriteRequestPB?

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            aserbin Alexey Serbin
            aserbin Alexey Serbin
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment