Uploaded image for project: 'Apache Hudi'
  1. Apache Hudi
  2. HUDI-526

inline compact not work

    XMLWordPrintableJSON

Details

    Description

      hoodie.compact.inline set as true

      hoodie.index.type set as INMEMEORY

       

      compact not occur after dela commit

      
      20/01/13 16:43:43 INFO HoodieMergeOnReadTable: Checking if compaction needs to be run on file:///tmp/hudi_cow_table_read
      20/01/13 16:43:43 INFO HoodieMergeOnReadTable: Compacting merge on read table file:///tmp/hudi_cow_table_read
      20/01/13 16:43:43 INFO FileSystemViewManager: Creating InMemory based view for basePath file:///tmp/hudi_cow_table_read
      20/01/13 16:43:43 INFO HoodieTableMetaClient: Loading HoodieTableMetaClient from file:///tmp/hudi_cow_table_read
      20/01/13 16:43:43 INFO FSUtils: Hadoop Configuration: fs.defaultFS: [file:///], Config:[Configuration: core-default.xml, core-site.xml, mapred-default.xml, mapred-site.xml, yarn-default.xml, yarn-site.xml, hdfs-default.xml, hdfs-site.xml, __spark_hadoop_conf__.xml], FileSystem: [org.apache.hadoop.fs.LocalFileSystem@6a24b9e2]
      20/01/13 16:43:43 INFO HoodieTableConfig: Loading table properties from file:/tmp/hudi_cow_table_read/.hoodie/hoodie.properties
      20/01/13 16:43:43 INFO HoodieTableMetaClient: Finished Loading Table of type MERGE_ON_READ(version=org.apache.hudi.common.model.TimelineLayoutVersion@20) from file:///tmp/hudi_cow_table_read
      20/01/13 16:43:43 INFO HoodieTableMetaClient: Loading Active commit timeline for file:///tmp/hudi_cow_table_read
      20/01/13 16:43:43 INFO HoodieActiveTimeline: Loaded instants [[20200109181330__deltacommit__COMPLETED], [20200110170000__deltacommit__COMPLETED], [20200110171526__deltacommit__COMPLETED], [20200113105844__deltacommit__COMPLETED], [20200113145851__deltacommit__COMPLETED], [20200113155502__deltacommit__COMPLETED], [20200113164342__deltacommit__COMPLETED]]
      20/01/13 16:43:43 INFO HoodieRealtimeTableCompactor: Compacting file:///tmp/hudi_cow_table_read with commit 20200113164343
      20/01/13 16:43:43 INFO HoodieTableMetaClient: Loading HoodieTableMetaClient from file:///tmp/hudi_cow_table_read
      20/01/13 16:43:43 INFO FSUtils: Hadoop Configuration: fs.defaultFS: [file:///], Config:[Configuration: core-default.xml, core-site.xml, mapred-default.xml, mapred-site.xml, yarn-default.xml, yarn-site.xml, hdfs-default.xml, hdfs-site.xml, __spark_hadoop_conf__.xml], FileSystem: [org.apache.hadoop.fs.LocalFileSystem@6a24b9e2]
      20/01/13 16:43:43 INFO HoodieTableConfig: Loading table properties from file:/tmp/hudi_cow_table_read/.hoodie/hoodie.properties
      20/01/13 16:43:43 INFO HoodieTableMetaClient: Finished Loading Table of type MERGE_ON_READ(version=org.apache.hudi.common.model.TimelineLayoutVersion@20) from file:///tmp/hudi_cow_table_read
      20/01/13 16:43:43 INFO HoodieTableMetaClient: Loading Active commit timeline for file:///tmp/hudi_cow_table_read
      20/01/13 16:43:43 INFO HoodieActiveTimeline: Loaded instants [[20200109181330__deltacommit__COMPLETED], [20200110170000__deltacommit__COMPLETED], [20200110171526__deltacommit__COMPLETED], [20200113105844__deltacommit__COMPLETED], [20200113145851__deltacommit__COMPLETED], [20200113155502__deltacommit__COMPLETED], [20200113164342__deltacommit__COMPLETED]]
      
      

       

      not compact time record in the .hoodie path

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              liujianhuiouc liujianhui
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 20m
                  20m