Uploaded image for project: 'Flume'
  1. Flume
  2. FLUME-2653

Allow inUseSuffix to be null/empty

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Needs documentation update for the additional parameter.

    Description

      At the moment, it doesn't seem possible to set the null/empty. We've tried '' which just adds the quotes to the end, and setting to nothing, which just uses the default .tmp.

      We want the in use file to have the same name as the closed file, so we can read from files that are in use without the file moving from underneath us. In our use case, we know that an in use file is still readable and parseable, because it is just text with a JSON document per line.

      It looks like the HDFS sink code can handle this change already, but at the moment there is no way to set the bucketPath and targetPath to be the same.

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            emajor Endre Major
            andrewrjones Andrew Jones
            Votes:
            0 Vote for this issue
            Watchers:
            12 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment