Uploaded image for project: 'Flume'
  1. Flume
  2. FLUME-2514

Some TestFileChannelRestart tests are extremely slow

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.7.0
    • Component/s: File Channel
    • Labels:
      None

      Description

      TestFileChannelRestart tests are really slow. For example, testToggleCheckpointCompressionFromFalseToTrue and testToggleCheckpointCompressionFromTrueToFalse take ~4 minutes each.

      Some of them could be made faster by using channels with lower capacity.

      1. FLUME-2215-0.patch
        1 kB
        Santiago M. Mola
      2. FLUME-2215-1.patch
        1 kB
        Denes Arvay

        Activity

        Hide
        bessbd Bessenyei Balázs Donát added a comment - - edited

        +1

        All tests remain passing, it actually makes them run faster.
        Collaborators: Denes Arvay, Attila Simon

        Show
        bessbd Bessenyei Balázs Donát added a comment - - edited +1 All tests remain passing, it actually makes them run faster. Collaborators: Denes Arvay , Attila Simon
        Hide
        mpercy Mike Percy added a comment -

        Thank you for the review, Bessenyei Balázs Donát. And thanks for the rebase, Denes Arvay.

        +1. I am about to commit this.

        Show
        mpercy Mike Percy added a comment - Thank you for the review, Bessenyei Balázs Donát . And thanks for the rebase, Denes Arvay . +1. I am about to commit this.
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit c554ba3d9b8234b37ba0b5dde3657034d11acb0b in flume's branch refs/heads/trunk from Santiago M. Mola
        [ https://git-wip-us.apache.org/repos/asf?p=flume.git;h=c554ba3 ]

        FLUME-2514. Speed up TestFileChannelRestart tests

        TestFileChannelRestart tests are really slow. For example,
        testToggleCheckpointCompressionFromFalseToTrue and
        testToggleCheckpointCompressionFromTrueToFalse take ~4 minutes each.

        Some of them could be made faster by using channels with lower capacity.

        Reviewed by Bessenyei Balázs Donát

        (Santiago M. Mola via Mike Percy)

        Show
        jira-bot ASF subversion and git services added a comment - Commit c554ba3d9b8234b37ba0b5dde3657034d11acb0b in flume's branch refs/heads/trunk from Santiago M. Mola [ https://git-wip-us.apache.org/repos/asf?p=flume.git;h=c554ba3 ] FLUME-2514 . Speed up TestFileChannelRestart tests TestFileChannelRestart tests are really slow. For example, testToggleCheckpointCompressionFromFalseToTrue and testToggleCheckpointCompressionFromTrueToFalse take ~4 minutes each. Some of them could be made faster by using channels with lower capacity. Reviewed by Bessenyei Balázs Donát (Santiago M. Mola via Mike Percy)
        Hide
        mpercy Mike Percy added a comment -

        Pushed to trunk. Thank you for the patch, Santiago M. Mola!

        Show
        mpercy Mike Percy added a comment - Pushed to trunk. Thank you for the patch, Santiago M. Mola !
        Hide
        hudson Hudson added a comment -

        UNSTABLE: Integrated in Flume-trunk-hbase-1 #183 (See https://builds.apache.org/job/Flume-trunk-hbase-1/183/)
        FLUME-2514. Speed up TestFileChannelRestart tests (mpercy: http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git&a=commit&h=c554ba3d9b8234b37ba0b5dde3657034d11acb0b)

        • flume-ng-channels/flume-file-channel/src/test/java/org/apache/flume/channel/file/TestFileChannelRestart.java
        Show
        hudson Hudson added a comment - UNSTABLE: Integrated in Flume-trunk-hbase-1 #183 (See https://builds.apache.org/job/Flume-trunk-hbase-1/183/ ) FLUME-2514 . Speed up TestFileChannelRestart tests (mpercy: http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git&a=commit&h=c554ba3d9b8234b37ba0b5dde3657034d11acb0b ) flume-ng-channels/flume-file-channel/src/test/java/org/apache/flume/channel/file/TestFileChannelRestart.java
        Hide
        liorze Lior Zeno added a comment -

        Mike Percy, do you manually resolve issues after commit or is there an automatic background process that does it?

        Show
        liorze Lior Zeno added a comment - Mike Percy , do you manually resolve issues after commit or is there an automatic background process that does it?
        Hide
        mpercy Mike Percy added a comment -

        Lior Zeno: I do it manually. Just forgot with this one. Thanks for noticing.

        The only automatic thing I do sometimes is close pull requests with something like "This closes #52" in the commit message.

        Marking this JIRA Fixed.

        Show
        mpercy Mike Percy added a comment - Lior Zeno : I do it manually. Just forgot with this one. Thanks for noticing. The only automatic thing I do sometimes is close pull requests with something like "This closes #52" in the commit message. Marking this JIRA Fixed.

          People

          • Assignee:
            smolav Santiago M. Mola
            Reporter:
            smolav Santiago M. Mola
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development