Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-10557

Allow Impala to use Kudu's multi-row transaction implementation

Attach filesAttach ScreenshotAdd voteVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Clients
    • Labels:
      None
    • Epic Color:
      ghx-label-12

      Description

      Once Kudu's support for multi-row transactions materializes, it'd be great if Impala could expose a query option or session-level flag to indicate that Impala should run queries that insert into Kudu in the context of a single transaction.

      When the query option is set, similar to how transactions are opened for Hive ACID tables, a new Kudu transaction should be started by Impala's Frontend, and the resulting KuduTransaction object should be serialized and passed as context to the Impala executors, which should open their Kudu sessions using the KuduTransaction::CreateSession() API instead of the usual KuduClient::NewSession().

      Errors returned in the session should be propagated to the Impala Frontend before committing – it seems reasonable to expect that any session errors should result in Impala aborting the transaction.

      The KuduTransaction handle in the frontend has a heartbeater built in, so unlike Hive ACID transactions, which require manual tracking of heartbeating, no additional manual requests are needed to maintain the transaction.

      Kudu's Java client API is defined in this patch, and C++ client API is defined in this patch. The implementation isn't fully operational quite yet – once this patch to automatically register participants lands, Impala developers should at least be able to start adding basic testing non-overlapping cases (partition locking is still in progress).

        Attachments

        Issue Links

          Activity

            People

            • Assignee:
              wzhou Wenzhe Zhou
              Reporter:
              awong Andrew Wong

              Dates

              • Created:
                Updated:

                Issue deployment