Uploaded image for project: 'Flume'
  1. Flume
  2. FLUME-2773

TailDirSource throws FileNotFound Exception if ~/.flume directory is not created already

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.7.0
    • Fix Version/s: 1.7.0
    • Component/s: None
    • Labels:
      None

      Description

      If we leave the positionFile parameter to default,
      then the following exception is thrown when ~/.flume is not present
      [We should take care of creating the default directory if not present ]

      2015-08-27 09:44:30,551 (positionWriter) [ERROR - org.apache.flume.source.taildir.TaildirSource.writePosition(TaildirSource.java:312)] Failed writing positionFile
      java.io.FileNotFoundException: /Users/jrufus/.flume/taildir_position.json (No such file or directory)
      at java.io.FileOutputStream.open(Native Method)
      at java.io.FileOutputStream.<init>(FileOutputStream.java:221)
      at java.io.FileOutputStream.<init>(FileOutputStream.java:171)
      at java.io.FileWriter.<init>(FileWriter.java:90)
      at org.apache.flume.source.taildir.TaildirSource.writePosition(TaildirSource.java:306)
      at org.apache.flume.source.taildir.TaildirSource.access$600(TaildirSource.java:56)
      at org.apache.flume.source.taildir.TaildirSource$PositionWriterRunnable.run(TaildirSource.java:298)

        Activity

        Hide
        jrufus Johny Rufus added a comment -

        Attached patch, that makes sure to create any missing parent directories in the positionFile path (either default or user given input)
        This ensures that position files are always created [Currently if default postionPath is assumed and .flume is not there, then the position checkpointing does not happen and user is not made aware of this fact, unless the user looks in the log file for the exception]

        Show
        jrufus Johny Rufus added a comment - Attached patch, that makes sure to create any missing parent directories in the positionFile path (either default or user given input) This ensures that position files are always created [Currently if default postionPath is assumed and .flume is not there, then the position checkpointing does not happen and user is not made aware of this fact, unless the user looks in the log file for the exception]
        Hide
        iijima_satoshi Satoshi Iijima added a comment -

        +1. Looks good to me.

        Show
        iijima_satoshi Satoshi Iijima added a comment - +1. Looks good to me.
        Hide
        hshreedharan Hari Shreedharan added a comment -

        +1. LGTM

        Show
        hshreedharan Hari Shreedharan added a comment - +1. LGTM
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 3fccd241d92bc5b70d216a2111e28897cf93c5df in flume's branch refs/heads/trunk from Hari Shreedharan
        [ https://git-wip-us.apache.org/repos/asf?p=flume.git;h=3fccd24 ]

        FLUME-2773. TailDirSource throws FileNotFound Exception if ~/.flume directory is not created already

        (Johny Rufus via Hari)

        Show
        jira-bot ASF subversion and git services added a comment - Commit 3fccd241d92bc5b70d216a2111e28897cf93c5df in flume's branch refs/heads/trunk from Hari Shreedharan [ https://git-wip-us.apache.org/repos/asf?p=flume.git;h=3fccd24 ] FLUME-2773 . TailDirSource throws FileNotFound Exception if ~/.flume directory is not created already (Johny Rufus via Hari)
        Hide
        hshreedharan Hari Shreedharan added a comment -

        Committed! Thanks Johny!

        Show
        hshreedharan Hari Shreedharan added a comment - Committed! Thanks Johny!
        Hide
        hudson Hudson added a comment -

        UNSTABLE: Integrated in Flume-trunk-hbase-1 #125 (See https://builds.apache.org/job/Flume-trunk-hbase-1/125/)
        FLUME-2773. TailDirSource throws FileNotFound Exception if ~/.flume directory is not created already (hshreedharan: http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git&a=commit&h=3fccd241d92bc5b70d216a2111e28897cf93c5df)

        • flume-ng-sources/flume-taildir-source/src/main/java/org/apache/flume/source/taildir/TaildirSource.java
        Show
        hudson Hudson added a comment - UNSTABLE: Integrated in Flume-trunk-hbase-1 #125 (See https://builds.apache.org/job/Flume-trunk-hbase-1/125/ ) FLUME-2773 . TailDirSource throws FileNotFound Exception if ~/.flume directory is not created already (hshreedharan: http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git&a=commit&h=3fccd241d92bc5b70d216a2111e28897cf93c5df ) flume-ng-sources/flume-taildir-source/src/main/java/org/apache/flume/source/taildir/TaildirSource.java

          People

          • Assignee:
            jrufus Johny Rufus
            Reporter:
            jrufus Johny Rufus
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development