Uploaded image for project: 'Apache Storm'
  1. Apache Storm
  2. STORM-2425

Storm Hive Bolt not closing open transactions

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.0.0, 1.x
    • Fix Version/s: 2.0.0, 1.1.0
    • Component/s: None
    • Labels:
      None

      Description

      Hive bolt will close connection only if parameter "max_connections" is exceeded or bolt dies. So if we open a connection to Hive via Hive bolt and some time later we stop producing messages to Hive bolt, connection will be maintained and corresponding transactions will be opened. This can be a problem if we launch two topologies and one of them will maintain open transactions doing nothing, and other will work writing messages to hive. At some point hive will launch compactions to collapse small delta files generated by Hive Bolt into one base file. But compaction wont launch if we have opened transactions.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                arunmahadevan Arun Mahadevan
                Reporter:
                arunmahadevan Arun Mahadevan
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 40m
                  40m