Flume
  1. Flume
  2. FLUME-1824

Inflights can complete successfully even if checkpoint fails

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: v1.4.0
    • Component/s: None
    • Labels:
      None

      Description

      In FlumeEventQueue class if a failure happens between:

          inflightPuts.serializeAndWrite();
          inflightTakes.serializeAndWrite();
      

      and

          backingStore.checkpoint();
          return true;
      

      the channel will not detect an error during replay, but will startup with inflights with correct checksums, but is invalid for the checkpoint.

      1. FLUME-1824.patch
        3 kB
        Hari Shreedharan

        Issue Links

          Activity

          Hari Shreedharan created issue -
          Hari Shreedharan made changes -
          Field Original Value New Value
          Attachment FLUME-1824.patch [ 12563699 ]
          Hari Shreedharan made changes -
          Remote Link This issue links to "Review (Web Link)" [ 11954 ]
          Hide
          Brock Noland added a comment -

          Committed to trunk and 1.4! Thanks!

          Show
          Brock Noland added a comment - Committed to trunk and 1.4! Thanks!
          Brock Noland made changes -
          Status Open [ 1 ] Resolved [ 5 ]
          Fix Version/s v1.4.0 [ 12323372 ]
          Resolution Fixed [ 1 ]
          Hide
          Hudson added a comment -

          Integrated in flume-trunk #349 (See https://builds.apache.org/job/flume-trunk/349/)
          FLUME-1824: Inflights can complete successfully even if checkpoint fails (Revision 1a2e0d7a7629eb32821c780d48a2c17f4e76a59e)

          Result = SUCCESS
          brock : http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git&a=commit&h=1a2e0d7a7629eb32821c780d48a2c17f4e76a59e
          Files :

          • flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/EventQueueBackingStore.java
          • flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/FlumeEventQueue.java
          • flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/EventQueueBackingStoreFile.java
          Show
          Hudson added a comment - Integrated in flume-trunk #349 (See https://builds.apache.org/job/flume-trunk/349/ ) FLUME-1824 : Inflights can complete successfully even if checkpoint fails (Revision 1a2e0d7a7629eb32821c780d48a2c17f4e76a59e) Result = SUCCESS brock : http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git&a=commit&h=1a2e0d7a7629eb32821c780d48a2c17f4e76a59e Files : flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/EventQueueBackingStore.java flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/FlumeEventQueue.java flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/EventQueueBackingStoreFile.java
          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          Open Open Resolved Resolved
          11h 58m 1 Brock Noland 08/Jan/13 17:02

            People

            • Assignee:
              Hari Shreedharan
              Reporter:
              Hari Shreedharan
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development