Flume
  1. Flume
  2. FLUME-1320

Add safeguard for checkpoint corruption detection

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: v1.2.0
    • Component/s: None
    • Labels:
      None

      Description

      The checkpoint process can potentially be interrupted by a catastrophic failure such as complete power loss leaving the checkpoint corrupted. When the channel is restarted, the (corrupted) checkpoint will get loaded and there could be dataloss as a result of this.

      It is therefore necessary to have a safeguard to detect if a checkpoint succeeded or failed.

      1. FLUME-1320-4.patch
        28 kB
        Arvind Prabhakar

        Activity

        Arvind Prabhakar created issue -
        Arvind Prabhakar made changes -
        Field Original Value New Value
        Attachment FLUME-1320-4.patch [ 12533447 ]
        Hide
        Arvind Prabhakar added a comment -

        Attached a patch that addresses this issue.

        Show
        Arvind Prabhakar added a comment - Attached a patch that addresses this issue.
        Arvind Prabhakar made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Fix Version/s v1.2.0 [ 12320243 ]
        Hide
        Mike Percy added a comment -

        Patch committed. Thanks Arvind!

        Show
        Mike Percy added a comment - Patch committed. Thanks Arvind!
        Mike Percy made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Hide
        Hudson added a comment -

        Integrated in flume-trunk #258 (See https://builds.apache.org/job/flume-trunk/258/)
        FLUME-1320. Add safeguard for checkpoint corruption detection.

        (Arvind Prabhakar via Mike Percy) (Revision 1353888)

        Result = UNSTABLE
        mpercy : http://svn.apache.org/viewvc/?view=rev&rev=1353888
        Files :

        • /incubator/flume/trunk/flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/FileChannel.java
        • /incubator/flume/trunk/flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/FlumeEventQueue.java
        • /incubator/flume/trunk/flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/Log.java
        • /incubator/flume/trunk/flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/LogFile.java
        • /incubator/flume/trunk/flume-ng-channels/flume-file-channel/src/test/java/org/apache/flume/channel/file/TestCheckpoint.java
        • /incubator/flume/trunk/flume-ng-channels/flume-file-channel/src/test/java/org/apache/flume/channel/file/TestFileChannel.java
        • /incubator/flume/trunk/flume-ng-channels/flume-file-channel/src/test/java/org/apache/flume/channel/file/TestFlumeEventQueue.java
        Show
        Hudson added a comment - Integrated in flume-trunk #258 (See https://builds.apache.org/job/flume-trunk/258/ ) FLUME-1320 . Add safeguard for checkpoint corruption detection. (Arvind Prabhakar via Mike Percy) (Revision 1353888) Result = UNSTABLE mpercy : http://svn.apache.org/viewvc/?view=rev&rev=1353888 Files : /incubator/flume/trunk/flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/FileChannel.java /incubator/flume/trunk/flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/FlumeEventQueue.java /incubator/flume/trunk/flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/Log.java /incubator/flume/trunk/flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/LogFile.java /incubator/flume/trunk/flume-ng-channels/flume-file-channel/src/test/java/org/apache/flume/channel/file/TestCheckpoint.java /incubator/flume/trunk/flume-ng-channels/flume-file-channel/src/test/java/org/apache/flume/channel/file/TestFileChannel.java /incubator/flume/trunk/flume-ng-channels/flume-file-channel/src/test/java/org/apache/flume/channel/file/TestFlumeEventQueue.java
        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Open Open Patch Available Patch Available
        49m 37s 1 Arvind Prabhakar 26/Jun/12 09:20
        Patch Available Patch Available Resolved Resolved
        1h 5m 1 Mike Percy 26/Jun/12 10:25

          People

          • Assignee:
            Arvind Prabhakar
            Reporter:
            Arvind Prabhakar
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development