[IGNITE-15557] [Ignite 3] Replace "changers" with "pojos" as a default mean to work with configurations

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 3.0.0-alpha2
• Labels: iep-55 ignite-3
• Epic Link: Configuration
• Assignee: Unassigned
• Reporter: Ivan Bessonov
• Created: 22/Sep/21 10:15
• Updated: 22/Sep/21 10:37

Description

Using lambda expressions to change Ignite configuration values proved to be inconvenient to many developers. Not only that, but using lambdas in Java thin client looks out of place and creates technical difficulties of properly serializing them. There's a request to use POJO classes, possibly with Builders associated to them.

New API

There's a number of ways one can approach the design of this API. I see two main options:

• good old pojo-based fluent interfaces:

ignite().nodeConfiguration()
    .setNetworkConfiguration(new NetworkConfig()
        .setServer(new ServerConfig()
            .setPortRange(1000)
        )
        .setNodeFinder(new StaticNodeFinderConfig() // polymorphic
            .setNetClusterNodes("localhost")
        )
    );

• same thing, but with "build()" at the end of each instance creation. Personally, I think it's a bit excessive here.

Somewhat challenging here is a notion of "named lists". I propose the following solution: 

new TableConfig()
    .addColumn(new ColumnConfiguration("col1").setType(INT32))
    .addColumn(new ColumnConfiguration("col2").setType(INT64))
// just puts null in the collection. Same as in HOCON notation
    .removeColumn("col3");

Also, same as HOCON notation, there's no API to manipulate ordering of elements. That's the internal stuff. Instances, passed into new "set" methods, should be treated if they represent a JSON object.

Serialization

Contrary to what just have written, named list elements should be processed in the same order as they were added into an object, at least that's a desirable feature. "LinkedHashMap" allows us to achieve this, but then it must be properly serialized in thin protocol. The problem is that one can't just create JSON tree out of the object and then convert it back - JSON does not preserve ordering of the elements.

Default Java serialization can solve the problem, but interoperability with other languages will suffer. Custom serialization format should probably be used or we should just treat the object as "OrderedMap<String, Either<OrderedMap, ?/* leaf values */>>. This would make the most sense, we just have to make sure that "the other side" reads it into "LinkedHashMap", not the plain one.

Changers API should be hidden

"org.apache.ignite.configuration.ConfigurationTree#change" method should be hidden somewhere. Maybe in "DynamicConfiguration", that's not very important.

What's important is this:

• ConfigurationTree should have method "update" or "set" that accepts POJO;
• "change" should be removed;
• There should be a method somewhere like this:

public static <Change> Future<Void> change(
    ConfigurationTree<?, Change, ?> cfg,
    Consumer<Change> change
) { ... }

and it'll do all the magic. Of course, it should be located somewhere in internal classes.

Node start API

It would actually be convenient to start nodes in embedded mode with the ability to pass configuration as an object, not as a text file. For tests at least. So builder like this would be welcomed in my opinion:

Ignite ignite = Ignition.serverNode(name, workDir)
    .withNodeFinder(...)
    .withNetwork(...)
    .start();

Implementation notes

Code generation in annotation processor is obviously required. I consider this a simple part.

What's less obvious is a way to convert POJO into "changer" closure in the internals of configuration framework.

• Option one. Each POJO object will provide a single magic method to convert itself into a prefix tree. This is the simplest solution one can come up with but it pollutes the interface of configuration objects.
• Option two. Generate runtime converters that use var handles or something as fancy. Not too hard to implement, no separate methods are required in objects. Con - same conversion logic would have to be implemented both in thin client and in server, that's not very good.

I lean towards to option 1 as the simplest.