Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-4416

Return warning msgs/counts to clients for ignored Kudu warnings/conflict violations

Attach filesAttach ScreenshotAdd voteVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Open
    • Major
    • Resolution: Unresolved
    • Kudu_Impala
    • None
    • Clients

    Description

      DML operations should return the number of rows modified (which works for beeswax but not HS2, see IMPALA-1789). Kudu DML operations may also have warnings that result in rows not being inserted/modified/deleted, and it is important that that information can be exposed in a precise way because the DML behavior is already non-standard (i.e. because there aren't transactional semantics).

      E.g. an insert should be able to report:

      1. how many rows were inserted
      2. how many rows weren't inserted due to:
        1. duplicate key conflicts
        2. nulls in non-nullable cols
        3. rows w/ PKs that don't exist

      The number of rows inserted is exposed in beeswax but not hs2. The 'query warnings' are exposed when using the shell today because the shell can report warnings, but these warning counts may not be precise since the warning infrastructure used some bounded buffer.

      There are a few things that need to happen:

      1. these warnings should be accounted for in a more precise way, probably separately from the bounded error-reporting buffer
      2. they need to be exposed via beeswax/hs2, though there is not a good mechanism for doing this in the current APIs so this will need some thought and maybe be a breaking change / clients need to know how to handle.

      Related JIRAs (about num modified rows, not warning handing):
      IMPALA-1789: about HS2 reporting num modified rows
      IMPALA-3713: about UPDATE/DELETE/UPSERT returning num modified rows

      Attachments

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            Unassigned Unassigned
            mjacobs Matthew Jacobs

            Dates

              Created:
              Updated:

              Slack

                Issue deployment