Flume
  1. Flume
  2. FLUME-1893

File Channel could miss possible checkpoint corruption

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: v1.3.1
    • Fix Version/s: v1.4.0
    • Component/s: None
    • Labels:
      None

      Description

      If the first 4 bytes of the checkpoint are overwritten, then the current sanity check will miss it because we read the first long and then cast it to an int, missing the first 4 bytes when we check the version. First 4 bytes being corrupt could indicate further corruption in the checkpoint which cannot be verified because the remaining data consists mostly of pointers to files, which if not used are simply ignored by the replay mechanism

      1. FLUME-1893.patch
        7 kB
        Hari Shreedharan

        Issue Links

          Activity

          Hari Shreedharan created issue -
          Hari Shreedharan made changes -
          Field Original Value New Value
          Assignee Hari Shreedharan [ hshreedharan ]
          Hide
          Hari Shreedharan added a comment -

          Patch from RB

          Show
          Hari Shreedharan added a comment - Patch from RB
          Hari Shreedharan made changes -
          Attachment FLUME-1893.patch [ 12567698 ]
          Hari Shreedharan made changes -
          Remote Link This issue links to "Review (Web Link)" [ 11998 ]
          Hari Shreedharan made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Affects Version/s v1.3.1 [ 12323879 ]
          Hari Shreedharan made changes -
          Attachment FLUME-1893.patch [ 12567698 ]
          Hari Shreedharan made changes -
          Attachment FLUME-1893.patch [ 12567699 ]
          Hari Shreedharan made changes -
          Attachment FLUME-1893.patch [ 12567699 ]
          Hari Shreedharan made changes -
          Attachment FLUME-1893.patch [ 12567716 ]
          Hide
          Mike Percy added a comment -

          +1

          Show
          Mike Percy added a comment - +1
          Hide
          Mike Percy added a comment -

          Thanks for the patch Hari!

          Pushed to trunk & flume-1.4 branches.

          Show
          Mike Percy added a comment - Thanks for the patch Hari! Pushed to trunk & flume-1.4 branches.
          Mike Percy made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Fix Version/s v1.4.0 [ 12323372 ]
          Resolution Fixed [ 1 ]
          Hide
          Hudson added a comment -

          Integrated in flume-trunk #407 (See https://builds.apache.org/job/flume-trunk/407/)
          FLUME-1893. File Channel could miss possible checkpoint corruption. (Revision 7a6e3db005dfc7848eeb62b6e66b91dc00a7f726)

          Result = FAILURE
          mpercy : http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git&a=commit&h=7a6e3db005dfc7848eeb62b6e66b91dc00a7f726
          Files :

          • flume-ng-channels/flume-file-channel/src/test/java/org/apache/flume/channel/file/TestFileChannelRestart.java
          • flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/EventQueueBackingStoreFile.java
          • flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/FileChannel.java
          • flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/Log.java
          Show
          Hudson added a comment - Integrated in flume-trunk #407 (See https://builds.apache.org/job/flume-trunk/407/ ) FLUME-1893 . File Channel could miss possible checkpoint corruption. (Revision 7a6e3db005dfc7848eeb62b6e66b91dc00a7f726) Result = FAILURE mpercy : http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git&a=commit&h=7a6e3db005dfc7848eeb62b6e66b91dc00a7f726 Files : flume-ng-channels/flume-file-channel/src/test/java/org/apache/flume/channel/file/TestFileChannelRestart.java flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/EventQueueBackingStoreFile.java flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/FileChannel.java flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/Log.java

            People

            • Assignee:
              Hari Shreedharan
              Reporter:
              Hari Shreedharan
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development