[FLINK-34033] flink json supports raw type

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 1.19.0
• Fix Version/s: 1.20.0
• Component/s: Formats (JSON, Avro, Parquet, ORC, SequenceFile)
• Assignee: Unassigned
• Reporter: Jacky Lau
• Created: 09/Jan/24 04:20
• Updated: 11/Mar/24 12:44

Description

when user use es type nested/object, user can using complex type like ROW/ARRAY/MAP.

but it will not convenient when the object type is not fixed size like ROW. for example, user my user udf to produce those data and insert to es. we can supports RAW type

CREATE TABLE es_sink (
 `string` VARCHAR, 
  nested RAW('java.lang.Object', 'AEdvcmcuYXBhY2hlLmZsaW5rLmFwaS5qYXZhLnR5cGV1dGlscy5ydW50aW1lLmtyeW8uS3J5b1NlcmlhbGl6ZXJTbmFwc2hvdAAAAAIAEGphdmEubGFuZy5PYmplY3QAAATyxpo9cAAAAAIAEGphdmEubGFuZy5PYmplY3QBAAAAEgAQamF2YS5sYW5nLk9iamVjdAEAAAAWABBqYXZhLmxhbmcuT2JqZWN0AAAAAAApb3JnLmFwYWNoZS5hdnJvLmdlbmVyaWMuR2VuZXJpY0RhdGEkQXJyYXkBAAAAKwApb3JnLmFwYWNoZS5hdnJvLmdlbmVyaWMuR2VuZXJpY0RhdGEkQXJyYXkBAAAAtgBVb3JnLmFwYWNoZS5mbGluay5hcGkuamF2YS50eXBldXRpbHMucnVudGltZS5rcnlvLlNlcmlhbGl6ZXJzJER1bW15QXZyb1JlZ2lzdGVyZWRDbGFzcwAAAAEAWW9yZy5hcGFjaGUuZmxpbmsuYXBpLmphdmEudHlwZXV0aWxzLnJ1bnRpbWUua3J5by5TZXJpYWxpemVycyREdW1teUF2cm9LcnlvU2VyaWFsaXplckNsYXNzAAAE8saaPXAAAAAAAAAE8saaPXAAAAAA'),
  object RAW('java.lang.Object', 'AEdvcmcuYXBhY2hlLmZsaW5rLmFwaS5qYXZhLnR5cGV1dGlscy5ydW50aW1lLmtyeW8uS3J5b1NlcmlhbGl6ZXJTbmFwc2hvdAAAAAIAEGphdmEubGFuZy5PYmplY3QAAATyxpo9cAAAAAIAEGphdmEubGFuZy5PYmplY3QBAAAAEgAQamF2YS5sYW5nLk9iamVjdAEAAAAWABBqYXZhLmxhbmcuT2JqZWN0AAAAAAApb3JnLmFwYWNoZS5hdnJvLmdlbmVyaWMuR2VuZXJpY0RhdGEkQXJyYXkBAAAAKwApb3JnLmFwYWNoZS5hdnJvLmdlbmVyaWMuR2VuZXJpY0RhdGEkQXJyYXkBAAAAtgBVb3JnLmFwYWNoZS5mbGluay5hcGkuamF2YS50eXBldXRpbHMucnVudGltZS5rcnlvLlNlcmlhbGl6ZXJzJER1bW15QXZyb1JlZ2lzdGVyZWRDbGFzcwAAAAEAWW9yZy5hcGFjaGUuZmxpbmsuYXBpLmphdmEudHlwZXV0aWxzLnJ1bnRpbWUua3J5by5TZXJpYWxpemVycyREdW1teUF2cm9LcnlvU2VyaWFsaXplckNsYXNzAAAE8saaPXAAAAAAAAAE8saaPXAAAAAA'),
  PRIMARY KEY (`string`) NOT ENFORCED
) WITH
('connector'='elasticsearch'); 

and es is dependent on flink-json currently, so we can make flink-json supports RAW type