Flume
  1. Flume
  2. FLUME-1104

HDFS rolls the first file incorrectly

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: v1.2.0
    • Fix Version/s: v1.2.0
    • Component/s: Sinks+Sources
    • Labels:
      None

      Description

      The sink process() keep tracks of the buckets opened during the transaction. At the end of transaction, we need to flush all the buckets that has pending data. This is required in order to ensure that the data removed from channel should be safely in HDFS during commit.
      Currently the files are tracked only when they are created and also getting closed during the cleanup instead of flush.

      1. FLUME-1104-2.patch
        8 kB
        Prasad Mujumdar

        Activity

        Arvind Prabhakar made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Fix Version/s v1.2.0 [ 12320243 ]
        Resolution Fixed [ 1 ]
        Prasad Mujumdar made changes -
        Attachment FLUME-1104-2.patch [ 12521636 ]
        Prasad Mujumdar made changes -
        Field Original Value New Value
        Summary HDFS file roller creates causes the first file to roll too soon HDFS rolls the first file incorrectly
        Prasad Mujumdar created issue -

          People

          • Assignee:
            Prasad Mujumdar
            Reporter:
            Prasad Mujumdar
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development