Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-9238

There are some questions about BuckeingSink when sinking data to S3.

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Duplicate
    • 1.4.2
    • None
    • Connectors / FileSystem
    • None

    Description

      When flink is sinking data to S3,  there are some IOExceptions from fs. For some reasons I throw the exceptions, then the job will restart.  Sometimes，fs.rename fails and there are pending part files which are from the last job and finished part files, but the count  of records is not the same，slight difference. The "Exactly once" should not be the results.

      Attachments

        Issue Links

          duplicates

          New Feature - A new feature of the product, which has yet to be developed. FLINK-6306 Sink for eventually consistent file systems

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved

          Activity

            People

              Unassigned Unassigned
              liuzx32 liuzhixin
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: