Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-26116 FLIP-188: Implement Table Store V0.1
  3. FLINK-27070

Reuse FileFormat instead of DecodingFormat/EncodingFormat to ensure thread safety.

Attach filesAttach ScreenshotVotersWatch issueWatchersLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    Description

      When testing table store I'm faced with the following warning messages:

      org.apache.org.impl.MemoryManagerImpl [] - Owner thread expected Thread[Writer -> Local Committer (1/16)#0,5,Flink Task Threads], got Thread[pool-8-thread-1,5,Flink Task Threads]
      

      This is because OrcBulkWriterFactory is not thread safe (it keeps WriterOptions as a class member and creates writers from the same WriterOptions object, which is not thread safe).

      What we should do is to reuse FileFormat instead of DecodingFormat/EncodingFormat to ensure thread safety.

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            TsReaper Caizhi Weng
            TsReaper Caizhi Weng
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment