Details
-
Sub-task
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
None
-
None
-
None
-
ghx-label-2
Description
'OPTIMIZE TABLE <table_name>' rewrites all files of the table regardless of size and type, even if the table does not contain any small or delete files.
With 'FILE_SIZE_THRESHOLD' option, the user should be able to specify a file size limit to rewrite only small files.
Syntax: OPTIMIZE TABLE <table_name> (FILE_SIZE_THRESHOLD_MB=100);
The value of the threshold is the file size in MBs. Data files larger than the given limit will only be rewritten if they are referenced from delete deltas.
Note that if 'FILE_SIZE_THRESHOLD' is set, only the selected files will be rewritten according to the latest schema and partition spec. Therefore the intact data files might still have an older schema or partition layout. Use 'OPTIMIZE TABLE table_name' to rewrite the entire table according to the latest schema and partititon layout.