Flume
  1. Flume
  2. FLUME-2514

Some TestFileChannelRestart tests are extremely slow

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: v1.7.0
    • Component/s: File Channel
    • Labels:
      None

      Description

      TestFileChannelRestart tests are really slow. For example, testToggleCheckpointCompressionFromFalseToTrue and testToggleCheckpointCompressionFromTrueToFalse take ~4 minutes each.

      Some of them could be made faster by using channels with lower capacity.

      1. FLUME-2215-0.patch
        1 kB
        Santiago M. Mola
      2. FLUME-2215-1.patch
        1 kB
        Denes Arvay

        Activity

        Hide
        Bessenyei Balázs Donát added a comment - - edited

        +1

        All tests remain passing, it actually makes them run faster.
        Collaborators: Denes Arvay, Attila Simon

        Show
        Bessenyei Balázs Donát added a comment - - edited +1 All tests remain passing, it actually makes them run faster. Collaborators: Denes Arvay , Attila Simon
        Hide
        Mike Percy added a comment -

        Thank you for the review, Bessenyei Balázs Donát. And thanks for the rebase, Denes Arvay.

        +1. I am about to commit this.

        Show
        Mike Percy added a comment - Thank you for the review, Bessenyei Balázs Donát . And thanks for the rebase, Denes Arvay . +1. I am about to commit this.
        Hide
        ASF subversion and git services added a comment -

        Commit c554ba3d9b8234b37ba0b5dde3657034d11acb0b in flume's branch refs/heads/trunk from Santiago M. Mola
        [ https://git-wip-us.apache.org/repos/asf?p=flume.git;h=c554ba3 ]

        FLUME-2514. Speed up TestFileChannelRestart tests

        TestFileChannelRestart tests are really slow. For example,
        testToggleCheckpointCompressionFromFalseToTrue and
        testToggleCheckpointCompressionFromTrueToFalse take ~4 minutes each.

        Some of them could be made faster by using channels with lower capacity.

        Reviewed by Bessenyei Balázs Donát

        (Santiago M. Mola via Mike Percy)

        Show
        ASF subversion and git services added a comment - Commit c554ba3d9b8234b37ba0b5dde3657034d11acb0b in flume's branch refs/heads/trunk from Santiago M. Mola [ https://git-wip-us.apache.org/repos/asf?p=flume.git;h=c554ba3 ] FLUME-2514 . Speed up TestFileChannelRestart tests TestFileChannelRestart tests are really slow. For example, testToggleCheckpointCompressionFromFalseToTrue and testToggleCheckpointCompressionFromTrueToFalse take ~4 minutes each. Some of them could be made faster by using channels with lower capacity. Reviewed by Bessenyei Balázs Donát (Santiago M. Mola via Mike Percy)
        Hide
        Mike Percy added a comment -

        Pushed to trunk. Thank you for the patch, Santiago M. Mola!

        Show
        Mike Percy added a comment - Pushed to trunk. Thank you for the patch, Santiago M. Mola !
        Hide
        Hudson added a comment -

        UNSTABLE: Integrated in Flume-trunk-hbase-1 #183 (See https://builds.apache.org/job/Flume-trunk-hbase-1/183/)
        FLUME-2514. Speed up TestFileChannelRestart tests (mpercy: http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git&a=commit&h=c554ba3d9b8234b37ba0b5dde3657034d11acb0b)

        • flume-ng-channels/flume-file-channel/src/test/java/org/apache/flume/channel/file/TestFileChannelRestart.java
        Show
        Hudson added a comment - UNSTABLE: Integrated in Flume-trunk-hbase-1 #183 (See https://builds.apache.org/job/Flume-trunk-hbase-1/183/ ) FLUME-2514 . Speed up TestFileChannelRestart tests (mpercy: http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git&a=commit&h=c554ba3d9b8234b37ba0b5dde3657034d11acb0b ) flume-ng-channels/flume-file-channel/src/test/java/org/apache/flume/channel/file/TestFileChannelRestart.java
        Hide
        Lior Zeno added a comment -

        Mike Percy, do you manually resolve issues after commit or is there an automatic background process that does it?

        Show
        Lior Zeno added a comment - Mike Percy , do you manually resolve issues after commit or is there an automatic background process that does it?
        Hide
        Mike Percy added a comment -

        Lior Zeno: I do it manually. Just forgot with this one. Thanks for noticing.

        The only automatic thing I do sometimes is close pull requests with something like "This closes #52" in the commit message.

        Marking this JIRA Fixed.

        Show
        Mike Percy added a comment - Lior Zeno : I do it manually. Just forgot with this one. Thanks for noticing. The only automatic thing I do sometimes is close pull requests with something like "This closes #52" in the commit message. Marking this JIRA Fixed.

          People

          • Assignee:
            Santiago M. Mola
            Reporter:
            Santiago M. Mola
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development