Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-15068

Disable RocksDB's local LOG by default

    XMLWordPrintableJSON

    Details

    • Release Note:
      Hide
      Logging in RocksDB (e.g., logging related to flush, compaction, memtable creation, etc.) has been disabled by default to prevent disk space from being filled up unexpectedly. Users that need to enable logging should implement their own RocksDBOptionsFactory that creates DBOptions instances with InfoLogLevel set to INFO_LEVEL.
      Show
      Logging in RocksDB (e.g., logging related to flush, compaction, memtable creation, etc.) has been disabled by default to prevent disk space from being filled up unexpectedly. Users that need to enable logging should implement their own RocksDBOptionsFactory that creates DBOptions instances with InfoLogLevel set to INFO_LEVEL.

      Description

      With Flink's default settings for RocksDB, it will write a log file (not the WAL, but pure logging statements) into the data folder. Besides periodic statistics, it will log compaction attempts, new memtable creations, flushes, etc.

      A few things to note about this practice:

      1. this LOG file is growing over time with no limit
      2. the default logging level is INFO
      3. the statistics in there may help looking into performance and/or disk space problems (but maybe you should be looking and monitoring metrics instead)
      4. this file is not useful for debugging errors since it will be deleted along with the local dir when the TM goes down

      With a custom OptionsFactory, the user can change the behaviour like the following:

          @Override
          public DBOptions createDBOptions(DBOptions currentOptions) {
              currentOptions = super.createDBOptions(currentOptions);
      
              currentOptions.setKeepLogFileNum(10);
              currentOptions.setInfoLogLevel(InfoLogLevel.WARN_LEVEL);
              currentOptions.setStatsDumpPeriodSec(0);
              currentOptions.setMaxLogFileSize(1024 * 1024); // 1 MB each
      
              return currentOptions;
          }

      However, the rotating logger does currently not work (it will not delete old log files - see https://github.com/dataArtisans/frocksdb/pull/12). Also, the user should not have to write his own OptionsFactory to get a sensible default.

      To prevent this file from filling up the disk, I propose to change Flink's default RocksDB settings so that the LOG file is effectively disabled (nothing is written to it by default).

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                NicoK Nico Kruber
                Reporter:
                NicoK Nico Kruber
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 20m
                  20m