Uploaded image for project: 'Chukwa'
  1. Chukwa
  2. CHUKWA-504

Implement ability to delete file in FileAdaptor after successfully uploaded

    Details

    • Type: New Feature
    • Status: In Progress
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: 0.4.0
    • Fix Version/s: None
    • Component/s: Data Collection
    • Labels:
      None

      Description

      My client's use case is to feed lots of little data files into the Chukwa system. They're collected first by a web API, saved to disk, and then a locally-running agent is notified with the path to the file name.

      Naturally, there's an issue of wasting disk space by keeping all the files around on the web nodes unnecessarily. So a simple addition of an optional "delete" parameter when adding a FileAdaptor appears to do the trick.

      Forthcoming is a patch against 0.4.0 that we used to achieve this. The parameter is optional in order to maintain compatibility.

        Activity

        Hide
        kirktrue Kirk True added a comment -

        Here's a patch to the FileAdaptor to delete the file after upload.

        Show
        kirktrue Kirk True added a comment - Here's a patch to the FileAdaptor to delete the file after upload.
        Hide
        asrabkin Ari Rabkin added a comment -

        I just committed the patch. Thanks, Kirk.

        Marking issue as in progress because I want to remind myself to fix the documentation to reflect this change.

        Show
        asrabkin Ari Rabkin added a comment - I just committed the patch. Thanks, Kirk. Marking issue as in progress because I want to remind myself to fix the documentation to reflect this change.

          People

          • Assignee:
            asrabkin Ari Rabkin
            Reporter:
            kirktrue Kirk True
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:

              Development