Details
-
Bug
-
Status: Open
-
Major
-
Resolution: Unresolved
-
master branch
-
None
-
None
Description
When loading TsFiles into IoTDB with the LOAD command, if some chunks in the files cross partitions of the loader, the chunks will be decoded and re-split according to the loader's parition interval.
Although the value encoding is recorded within the files in ChunkHeader or ChunkMetadata, the time encoding is not recorded anywhere.
The current implementation uses the loader's time encoding to decode time chunks, which may introduce inconsistency with the generator of the files.
For example, an edge server generates TsFiles with PLAIN as the time encoding (one possible reason is that the server lacks of computing resource for encoding) and uploads it to the cloud cluster, where the time encoding is TS_2DIFF.
As a result, the receiver cluster will try to decode not-encoded chunks with TS_2DIFF, and the result is unpredictable.
To avoid the encoding inconsistency, one simplest way may be specifying the time encoding in the LOAD command, like:
LOAD '/data/tsfiles_to_load/' time_encoding=PLAIN
The important part is how to deal with such inconsistency when detected.
The least-effort-taking method is to reject such a LOAD command if the provided parameter is inconsistent with the local setting.
One more friendly approach could be re-encoding the chunks by the receiver with its own encoding.
