[SOLR-7438] Look into using new HDFS truncate feature in HdfsTransactionLog.

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: Hadoop Integration, hdfs
• Assignee: Mark Miller
• Reporter: Mark Miller
• Created: 21/Apr/15 16:14
• Updated: 19/Mar/19 11:31

Description

Looks like truncate is added in 2.7.

See HdfsTransactionLog:

  // HACK
  // while waiting for HDFS-3107, instead of quickly
  // dropping, we slowly apply
  // This is somewhat brittle, but current usage
  // allows for it
  @Override
  public boolean dropBufferedUpdates() {
    Future<RecoveryInfo> future = applyBufferedUpdates();
    if (future != null) {
      try {
        future.get();
      } catch (InterruptedException | ExecutionException e) {
        throw new RuntimeException(e);
      }
    }
    return true;
  }

Issue Links

• is related to: SOLR-8094 HdfsUpdateLog should not replay buffered documents as a replacement to dropping them.
• is related to: SOLR-9076 Update to Hadoop 2.7.2