Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-16361

Automatically kill runaway client processes

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Critical
    • Resolution: Unresolved
    • Affects Version/s: 1.0.0
    • Fix Version/s: None
    • Component/s: Transactions
    • Labels:
      None

      Description

      HIVE-13249 added an enforceable limit on how many transactions can be opened concurrently where the system starts to reject new work to prevent the system getting to a point where it cannot manage the load.

      Another condition to guard against is a runaway process (which would usually be some app (e.g. Storm) using Streaming Ingest API) that create a very large number of transactions very quickly all of which immediately get aborted due to some misconfiguration. This can cause large amount of metatdata to accumulate in the ACID system slowing everything down and causing instability.

      Now that we have TXNS.TXN_AGENT_INFO information we could probably use that to refuse work from a client even before we open any txns if it passes some "runaway client" heuristic.

      This is like an unintentional DOS attack

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                ekoifman Eugene Koifman
              • Votes:
                0 Vote for this issue
                Watchers:
                0 Start watching this issue

                Dates

                • Created:
                  Updated: