[HBASE-27636] The "CREATE_TIME_TS" value of the hfile generated by the HFileOutputFormat2 class is 0

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.6.0, 3.0.0-alpha-4, 2.4.17, 2.5.4
• Component/s: HFile, mapreduce
• Hadoop Flags: Reviewed
• Assignee: selina.yan
• Reporter: selina.yan
• Due: 13/Feb/23
• Created: 13/Feb/23 12:36
• Updated: 19/Feb/23 05:39
• Resolved: 18/Feb/23 03:03

Description

When HFileOutputFormat2 is used to create an hfile, the CREATE_TIME_TS of hfileinfo will not be assigned when creating an hfileContext, resulting in the value of  lastMajorCompactionAge  is the timestamp of the current time.

##HFileOutputFormat2.class
HFileContextBuilder contextBuilder = new HFileContextBuilder()
                            .withCompression(compression)
                            .withChecksumType(HStore.getChecksumType(conf))
                            .withBytesPerCheckSum(HStore.getBytesPerChecksum(conf))
                            .withBlockSize(blockSize);

##get lastMajorCompactionTs metric

  lastMajorCompactionTs = this.region.getOldestHfileTs(true);
...
  long now = EnvironmentEdgeManager.currentTime();
  return now - lastMajorCompactionTs;
...
##
public long getOldestHfileTs(boolean majorCompactionOnly) throws IOException {
  long result = Long.MAX_VALUE;
  for (HStore store : stores.values()) {
    Collection<HStoreFile> storeFiles = store.getStorefiles();
   ...
    for (HStoreFile file : storeFiles) {
      StoreFileReader sfReader = file.getReader();
     ...
      result = Math.min(result, reader.getFileContext().getFileCreateTime());
    }
  }
  return result == Long.MAX_VALUE ? 0 : result;
}

 

 

Issue Links

• links to: GitHub Pull Request #5033
• links to: GitHub Pull Request #5034