Uploaded image for project: 'Flume'
  1. Flume
  2. FLUME-2804

Hive sink - abort remaining transactions on shutdown

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.6.0
    • Fix Version/s: 1.7.0
    • Component/s: Sinks+Sources
    • Labels:

      Description

      Currently the hive sink does not explicitly abort unused transactions. Although these eventually timeout on the hive side, it is preferable to explicitly abort them so that the associated locks on the hive table/partition are released. As long as the locks stay open, the table/partition cannot be dropped.

      1. FLUME-2804.patch
        4 kB
        Sriharsha Chintalapani

        Activity

        Hide
        roshan_naik Roshan Naik added a comment -

        +1

        Show
        roshan_naik Roshan Naik added a comment - +1
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit bd80c5e67deda4ec146f19fc92ff0c3bff61a982 in flume's branch refs/heads/trunk from Roshan Naik
        [ https://git-wip-us.apache.org/repos/asf?p=flume.git;h=bd80c5e ]

        FLUME-2804. Hive sink - abort remaining transactions on shutdown

        (Sriharsha Chintalapani via Roshan Naik)

        Show
        jira-bot ASF subversion and git services added a comment - Commit bd80c5e67deda4ec146f19fc92ff0c3bff61a982 in flume's branch refs/heads/trunk from Roshan Naik [ https://git-wip-us.apache.org/repos/asf?p=flume.git;h=bd80c5e ] FLUME-2804 . Hive sink - abort remaining transactions on shutdown (Sriharsha Chintalapani via Roshan Naik)
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 9ed11d169f9616dc4842f4ec5617a4431d1cf79a in flume's branch refs/heads/flume-1.7 from Roshan Naik
        [ https://git-wip-us.apache.org/repos/asf?p=flume.git;h=9ed11d1 ]

        FLUME-2804. Hive sink - abort remaining transactions on shutdown

        (Sriharsha Chintalapani via Roshan Naik)

        Show
        jira-bot ASF subversion and git services added a comment - Commit 9ed11d169f9616dc4842f4ec5617a4431d1cf79a in flume's branch refs/heads/flume-1.7 from Roshan Naik [ https://git-wip-us.apache.org/repos/asf?p=flume.git;h=9ed11d1 ] FLUME-2804 . Hive sink - abort remaining transactions on shutdown (Sriharsha Chintalapani via Roshan Naik)
        Hide
        roshan_naik Roshan Naik added a comment -

        Committed.

        Thanks Sriharsha Chintalapani

        Show
        roshan_naik Roshan Naik added a comment - Committed. Thanks Sriharsha Chintalapani
        Hide
        hudson Hudson added a comment -

        ABORTED: Integrated in Flume-trunk-hbase-1 #126 (See https://builds.apache.org/job/Flume-trunk-hbase-1/126/)
        FLUME-2804. Hive sink - abort remaining transactions on shutdown (roshan: http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git&a=commit&h=bd80c5e67deda4ec146f19fc92ff0c3bff61a982)

        • flume-ng-sinks/flume-hive-sink/src/main/java/org/apache/flume/sink/hive/HiveWriter.java
        • flume-ng-sinks/flume-hive-sink/src/main/java/org/apache/flume/sink/hive/HiveSink.java
        Show
        hudson Hudson added a comment - ABORTED: Integrated in Flume-trunk-hbase-1 #126 (See https://builds.apache.org/job/Flume-trunk-hbase-1/126/ ) FLUME-2804 . Hive sink - abort remaining transactions on shutdown (roshan: http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git&a=commit&h=bd80c5e67deda4ec146f19fc92ff0c3bff61a982 ) flume-ng-sinks/flume-hive-sink/src/main/java/org/apache/flume/sink/hive/HiveWriter.java flume-ng-sinks/flume-hive-sink/src/main/java/org/apache/flume/sink/hive/HiveSink.java

          People

          • Assignee:
            sriharsha Sriharsha Chintalapani
            Reporter:
            sriharsha Sriharsha Chintalapani
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development