[IGNITE-16691] Storage Configuration refactoring and improvements

• Type: Task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.0.0-alpha5
• Labels: iep-55 ignite-3
• Epic Link: Pluggable Storage
• Assignee: Kirill Tkalenko
• Reporter: Sergey Chugunov
• Reviewer: Ivan Bessonov
• Created: 15/Mar/22 11:41
• Updated: 15/Apr/22 09:54
• Resolved: 05/Apr/22 10:21

Description

Currently Storage configuration in Ignite 3.0 reuses concepts right from Ignite 2.x like DataRegion.

However there are substantial differences between 2.x and 3.0 versions: when there is only one option of Storage in 2.x (off-heap page-based storage), 3.0 supports multiple types of storage (right now RocksDB implementing LSM trees model, refactored page-based storage in the near future).

With that in mind we need to refactor current configuration storage to the most abstract way, so Storage configuration entity would have only its Type (e.g. LSM or PAGE_MEMORY) and human-readable Name (in one-to-one relationship with Type).

Other implementation details like DataRegion should be hidden in polymorphic configuration.

Implementation proposal:

• Each storage defines its own configuration (schema) if needed: for example, use a new configuration root.
• Instead of specifying the date region name for the table configuration (TableConfigurationSchema), the specific storage configuration (DataStorageConfigurationSchema) will be specified with all the required fields.
• Adding a new storage will go through adding a new module.
  • Adding an implementation of StorageEngineFactory#createEngine(ConfigurationRegistry configRegistry, Path partitionsStoreDir) through the java.util.ServiceLoader mechanism to create new engines.
  • Adding an heir (polymorphic instance) of DataStorageConfigurationSchema.
  • Adding an implementation of ConfigurationModule to define all required configuration, which is also loaded through java.util.ServiceLoader mechanism.
  • Adding an implementation of StorageEngine, it is important that StorageEngine#name be implemented so that it can be mapped to DataStorageConfigurationSchema#name to determine the desired storage.

API:

// Configuration schemas.

@Config
public class TableConfigurationSchema {
    @ConfigValue
    public DataStorageConfigurationSchema dataStorage;
	
	// Other fields.
}

@PolymorphicConfig
public class DataStorageConfigurationSchema {
    @PolymorphicId(hasDefault = true)
    public String name = "rocksdb";
}

// Interfaces.

public interface StorageEngineFactory {
    StorageEngine createEngine(ConfigurationRegistry clusterConfigRegistry, Path partitionsStoreDir) throws StorageException;
}

public interface StorageEngine {
    String name();
	
    TableStorage createTable(TableConfiguration tableCfg);
	
// Other methods.
}

// Classes.

public class DataStorageManager implements IgniteComponent {
	private final Map<String, StorageEngine> engines;

	public @Nullable StorageEngine engine(DataStorageConfiguration config) {
		returns engines.get(config.value().name());
	}
	
	// Other methods.
}

Implementation example for RocksDB (located in ignite-storage-rocksdb):

@ConfigurationRoot(rootName = RocksDbStorageEngine.ENGINE_NAME, type = DISTRIBUTED)
public class RocksDbStorageEngineConfigurationSchema {
    public static final String DEFAULT_DATA_REGION_NAME = "default";

    @Name(DEFAULT_DATA_REGION_NAME)
    @ConfigValue
    public RocksDbDataRegionConfigurationSchema defaultRegion;

    @ExceptKeys(DEFAULT_DATA_REGION_NAME)
    @NamedConfigValue
    public RocksDbDataRegionConfigurationSchema regions;
}

@PolymorphicConfigInstance(RocksDbStorageEngine.ENGINE_NAME)
public class RocksDbDataStorageConfigurationSchema extends DataStorageConfigurationSchema {
    @Immutable
    @Value(hasDefault = true)
    public String dataRegion = RocksDbStorageEngineConfigurationSchema.DEFAULT_DATA_REGION_NAME;
}

@Config
public class RocksDbDataRegionConfigurationSchema {
    public static final String ROCKSDB_LRU_CACHE = "lru";

    public static final String ROCKSDB_CLOCK_CACHE = "clock";

    @InjectedName
    public String name;
	
    @Value(hasDefault = true)
    public long size = 256 * 1024 * 1024;

    @Value(hasDefault = true)
    @Min(1)
    public long writeBufferSize = 64 * 1024 * 1024;

    @OneOf(ROCKSDB_LRU_CACHE)
    @Value(hasDefault = true)
    public String cache = ROCKSDB_LRU_CACHE;

    @Min(-1)
    @Value(hasDefault = true)
    public int numShardBits = -1;
}

public class RocksDbStorageEngineFactory implements StorageEngineFactory {
    @Override
    public StorageEngine createEngine(ConfigurationRegistry clusterConfigRegistry, Path partitionsStoreDir) throws StorageException {
        return new RocksDbStorageEngine(clusterConfigRegistry.getConfiguration(RocksDbStorageEngineConfiguration.KEY), partitionsStoreDir);
    }
}

public class RocksDbStorageEngine implements StorageEngine {
    public static final String ENGINE_NAME = "rocksdb";
	
    @Override
    public String name() {
        return ENGINE_NAME;
    }
	
    // Start of all regions and other methods.
}

Issue Links

• is a parent of: IGNITE-16737 Storage Engine interfaces unification
• is depended upon by: IGNITE-16765 Support creation of table for specific storage engine
• is depended upon by: IGNITE-16766 Support storage engine configuration parameters during table creation.
• relates to: IGNITE-16792 Configuration for Default Storage Engine
• relates to: IGNITE-16797 DDL support for Storage specific parameters
• relates to: IGNITE-16835 Add data storage support to TableDefinition (public API)
• links to: GitHub Pull Request #754