Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-21543

when using FIFO compaction, I found sst being deleted on the first checkpoint

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Labels:
      None

      Description

      2021/03/01-18:51:01.202049 7f59042fc700 (Original Log Time 2021/03/01-18:51:01.200883) [/compaction/compaction_picker_fifo.cc:107] [_timer_state/processing_user-timers] FIFO compaction: picking file 1710 with creation time 0 for deletion

       

      the configuration is like 

      currentOptions.setCompactionStyle(getCompactionStyle());
      currentOptions.setLevel0FileNumCompactionTrigger(8);
      // currentOptions.setMaxTableFilesSizeFIFO(MemorySize.parse("2gb").getBytes());
      CompactionOptionsFIFO compactionOptionsFIFO = new CompactionOptionsFIFO();
      compactionOptionsFIFO.setMaxTableFilesSize(MemorySize.parse("8gb").getBytes());
      compactionOptionsFIFO.setAllowCompaction(true);

       

      the rocksdb version is 

      <dependency>
      <groupId>io.github.myasuka</groupId>
      <artifactId>frocksdbjni</artifactId>
      <version>6.10.2-ververica-3.0</version>
      </dependency>

       

      I think the problem is caused by tableproperty is lost by snapshot. Can any one suggest how i can skip this problem?

        Attachments

        1. LOG (2)
          38.76 MB
          xiaogang zhou
        2. image-2021-03-03-13-09-01-695.png
          66 kB
          xiaogang zhou
        3. image-2021-03-03-11-35-11-458.png
          250 kB
          xiaogang zhou

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              zhoujira86 xiaogang zhou
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: