Chukwa
  1. Chukwa
  2. CHUKWA-454

DirTailingAdaptor Add option to Filter files.

    Details

    • Type: New Feature New Feature
    • Status: Resolved
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 0.4.0
    • Fix Version/s: 0.4.0
    • Component/s: Data Collection
    • Labels:
      None

      Description

      I would like to be able to point the DirTailingAdaptor at a directory but only have it tail files matching to a certain pattern.

      So I've made a change that allows the adaptor to recieve an optional filter parameter and use the org.apache.commons.io.filefilter.WildcardFileFilter class to filter out files.
      This allows you to for example point the adaptor at a tomcat log directory and not have it tail catalina.out logs etc.

      1. CHUKWA-454-v0.1.patch
        8 kB
        Gerrit Jansen van Vuuren

        Activity

        Hide
        Hudson added a comment -
        Show
        Hudson added a comment - Integrated in Chukwa-trunk #330 (See http://hudson.zones.apache.org/hudson/job/Chukwa-trunk/330/ )
        Hide
        Gerrit Jansen van Vuuren added a comment -

        No probs. Cheers.

        Show
        Gerrit Jansen van Vuuren added a comment - No probs. Cheers.
        Hide
        Ari Rabkin added a comment -

        Unit tests pass, and I am committing this. Thanks, Gerrit.

        Show
        Ari Rabkin added a comment - Unit tests pass, and I am committing this. Thanks, Gerrit.
        Hide
        Gerrit Jansen van Vuuren added a comment -

        Hi,

        I've made the changes, hope the spaces are 2 and not tab, I've configured eclipse to use only spaces and not tabs, checked also with sed for tab occurances didn't find any in the new patch.

        I hope you don't mind but I've extended the TestDirTailingAdaptor test with another method which is basically a copy of the existing test method, just with the extra lines of adding filters and a file that should be ignored by the filter. This assures that the filtering works properly.

        Show
        Gerrit Jansen van Vuuren added a comment - Hi, I've made the changes, hope the spaces are 2 and not tab, I've configured eclipse to use only spaces and not tabs, checked also with sed for tab occurances didn't find any in the new patch. I hope you don't mind but I've extended the TestDirTailingAdaptor test with another method which is basically a copy of the existing test method, just with the extra lines of adding filters and a file that should be ignored by the filter. This assures that the filtering works properly.
        Hide
        Gerrit Jansen van Vuuren added a comment -

        Extended TestDirTailingAdaptor.java to test filter functionality.
        Applied space changes.

        Show
        Gerrit Jansen van Vuuren added a comment - Extended TestDirTailingAdaptor.java to test filter functionality. Applied space changes.
        Hide
        Gerrit Jansen van Vuuren added a comment -

        Tomorrow I'll make some time to create a unit test for this feature, with the changes above also and resubmit the patch.

        Show
        Gerrit Jansen van Vuuren added a comment - Tomorrow I'll make some time to create a unit test for this feature, with the changes above also and resubmit the patch.
        Hide
        Ari Rabkin added a comment -

        This is something I was meaning to do, as well. If you have time, can you also put together one or more junit tests, to make sure this feature doesn't rot? They don't have to be super-sophisticated, I just want to make sure nobody breaks this inadvertently in development.

        Show
        Ari Rabkin added a comment - This is something I was meaning to do, as well. If you have time, can you also put together one or more junit tests, to make sure this feature doesn't rot? They don't have to be super-sophisticated, I just want to make sure nobody breaks this inadvertently in development.
        Hide
        Gerrit Jansen van Vuuren added a comment -

        Hi,
        yep will do.
        cheers.

        Show
        Gerrit Jansen van Vuuren added a comment - Hi, yep will do. cheers.
        Hide
        Eric Yang added a comment -

        The patch is fine, but could you generate another one with the same indent style (2 spaces)?
        Thanks

        Show
        Eric Yang added a comment - The patch is fine, but could you generate another one with the same indent style (2 spaces)? Thanks
        Hide
        Gerrit Jansen van Vuuren added a comment -

        This patch uses the commons Fileutils with the WildcardFileFilter to filter out files based on an optional argument.

        for(Object f: FileUtils.listFiles(dir, fileFilter, FileFilterUtils.trueFileFilter()))

        { scanDirHierarchy((File)f); }

        Unfortunately the commons.io package does not have generics in yet so I had to do Object then cast to File.

        And example of using it:

        add DirTailingAdaptor test somedir app*.log filetailer.CharFileTailingAdaptorUTF8 0

        Show
        Gerrit Jansen van Vuuren added a comment - This patch uses the commons Fileutils with the WildcardFileFilter to filter out files based on an optional argument. for(Object f: FileUtils.listFiles(dir, fileFilter, FileFilterUtils.trueFileFilter())) { scanDirHierarchy((File)f); } Unfortunately the commons.io package does not have generics in yet so I had to do Object then cast to File. And example of using it: add DirTailingAdaptor test somedir app*.log filetailer.CharFileTailingAdaptorUTF8 0

          People

          • Assignee:
            Unassigned
            Reporter:
            Gerrit Jansen van Vuuren
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development